Validate your code together with your peers and share your data safely to contribute to a traceable international model evaluation framework for the development of trustful wind energy engineering tools.
WEMEP addresses quality assurance of models being used for research and to drive wind energy applications. This is achieved through a framework to conduct formal verification and validation (V&V) that ultimately determines how model credibility is built upon. The protocol is based SANDIA’s V&V Framework[HMN15] which, itselft, is based on well-established procedures developed by various organizations including the Department of Energy, National Aeronautics and Space Administration, the American Institute of Aeronautics and Astronautics, and the American Society of Mechanical Engineers. Background articles include [OT02], [OTH04] and [OB06]. The framework’s primary focus is to provide guidance on the development and execution of tightly integrated modeling/experimental programs based on well-established V&V practices for the purpose of model assessment.
Based on the AIAA guide for V&V of computational fluid dynamics (CFD) [AIA98]:
Verification is the process of determining that the model implementation accurately represents the developer’s conceptual description of the model and the solution of the model. Here accuracy is measured with respect to benchmark solutions of simplified model problems
Validation is the process of determining the degree to which the model is an accurate representation of the real world from the perspective of the intended uses of the model. Here accuracy is measured with respect to experimental data.
The AIAA guide states that verification and validation are processes or ongoing activities without a clearly defined completion point. It is a matter of performing as many V&V exercises as possible in order to gain confidence and credibility on the model results towards the specific intended use of the model. Indeed, the intended use, i.e. the target application, is the main driver of this process to define the physical scope of the design system, its range of operating conditions, the variables of interest and their associated acceptance criteria. These acceptance criteria are defined in terms of error metrics that should be unified by the user community.
The intrinsic high complexity of the wind energy design system makes it very difficult to validate the full range of operating conditions. Hence, it is implicit that a validated model will use inference methodologies to extrapolate performance from the validation space to the operational space (Fig. 1). Therefore, the main objective of the validation process is to develop and quantify enough confidence on the computer model (or code) so that they can be used reliably to predict the quantities of interest within acceptable limits. Hence, validation is sometimes also referred to the assessment of the predictive capacity of a code.
Different scenarios of validation vs application space (adapted from [OTH04]).¶
Based in Europe, it is also worth mentioning the COST-732 Model Evaluation Guidance and Protocol Document[BS07] with focus on microscale modeling for the dispersion of pollutants in the urban environment. The protocol comprises the following aspects:
A scientific evaluation process, that considers the formulation of the models in terms of physics included and the degree of suitability for the intended use.
A verification process that addresses both the code (consistency with the conceptual model) and the solution procedure (to estimate the numerical error)
The provision of appropriate and quality assured validation datasets.
A model validation process in which model results are compared with experimental datasets.
An operational evaluation process that reflects the needs and responsibilities of the model user.
Quoting COST-732, models of whatever type are only of use if their quality (fitness-for-purpose) has been quantified, documented, and communicated to potential users[BS07]. Hence, WEMEP will define the framework that wind energy model developers can follow to make their codes trustfull for the wind energy community. Trust is built when the code performance has been tested and quantified based on appropriate datasets agreed upon to cover a relevant range of applicability. This protocol shall also support the planning, setting up and execution of forthcoming experiments that will feed the validation process as a sistematic and sustained activity for model development.
WEMEP is a community project with the following objectives:
To develop an international framework to guide model developers and end users on methodologies and best practices to conduct formal verification, validation and uncertainty quantification (VV&UQ).
To promote collaboration between modeling communities and foster interdisciplinary research and development towards integrated models.
To make model evaluation traceable through best practices for model evaluation and benchmarking and through open-access repositories of models, validation cases and data analysis scripts.
The protocol is defined in generic model-agnostic terms so it can be adopted by any modeling community. Then, each community can document their interpretation of the protocol in the definition of suitable validation strategies for the intended uses of their models. Ultimately, this results in the definition of a hierarchy of verification and validation cases of increasing complexity. These cases are curated by the community through model intercomparison benchmarks archived as public data repositories. Knowledge gaps identified in the V&V process are addressed by planning, setting up and executing targeted experiments.
The protocol is launched from the IEA Wind TCP Task 31 Wakebench which is focused on the evaluation of wind farm flow models. This includes models for the atmospheric boundary layer, to simulate wind conditions for wind resource and site suitability assessment, as well as wake models for the assessment of wind farm array efficiency and loads in connection to wind farm design.
Modeling communities are welcomed to implement the protocol and contribute with open access repositories that can be interopeable with those from other communities.
The most important keywords of the evaluation process are defined next, extracted from [HRCS13]. The purpose of this list is to adopt a common terminology when discussing model evaluation results. Terms are ordered alphabetically:
Benchmark: Typically in literature this is defined as an analytical or highly accurate numerical solution for use in verification [BS07]. However, this term is often being used to describe experimental datasets for use in validation, therefore care should be taken when using this term to clarify the accurateness and purpose of the dataset.
Blind test: Comparison of numerical results with experimental data, where modelers are not allowed access to the experimental dataset.
Error: Inaccuracy of the numerical model i.e., insufficient time-step resolution or spatial grid convergence. This can be known error due to limitations in implementing the mathematical equations (acknowledged error) or unknown error from mistakes (unacknowledged error).
Scientific evaluation: Determining the appropriateness of the conceptual model in describing the real world application, includes three parts: scientific review, verification and validation.
Extrapolation: Using a numerical model to simulate a process outside the range of which it was previously validated.
Conceptual model: System of mathematical equations, governing laws, initial and boundary conditions that describe the physical process of interest in the selected real world application.
Computational model: Implementation of the conceptual model into computer code.
Metric: Variable used to quantitatively compare results from a numerical model with experimental data, typically with specified criteria for validation.
Numerical calibration: Utilizing field measurements, ensuring the proper scaling and units, as input parameters to the numerical model that are not a priori known.
Numerical model: Another term for conceptual or computational model, this term is provided to distinguish between wind tunnel data and computer simulations.
Physical model: Non-numerical modeling of a real world process; i.e., using a wind tunnel or water tunnel to model a real world process to provide a high quality dataset for the validation of computational models.
Prediction: The output from a validated numerical simulation, for a specific real world process that is within the modeling capabilities deemed acceptable from the numerical model validation.
Quantity of interest: Output variable from numerical model to compare directly with experimental data, the metric is used to quantitatively compare the two results.
Real world: Determination of the physical process to be investigated, examples for wind energy applications include wind flow patterns and flow around a wind turbine.
Scientific review: The first step in model evaluation, it is an investigation of the scientific basis of a numerical model, which physical processes are included, how they are modeled, assumptions, approximations, solution techniques and the interface and resources available to the user.
Tuning: Making adjustments to parameters in the numerical model based on the comparison between the model output and field measurements, not considered orthodox validation since it is not a blind test.
Uncertainty: Recognizable inaccuracies of the model that are not due to a lack of knowledge. This can be due to inherent variability in the physical process (aleatory uncertainty) or from a lack of scientific understanding (epistemic uncertainty). Epistemic uncertainty can be improved by increasing modeling skill or understanding.
Validation: Ensuring the physical processes are accurately modeled, this involves a comparison of the computational results with experimental data.
Variability: In this case of wind energy this is the aleatory uncertainty attributed to the irregularity of turbulent processes in the atmosphere.
Verification: Ensuring the mathematical accuracy of the computational model, including accurate implementation of equations (Solution Verification) and checking the computer code for errors (Code Verification).
The building-block model evaluation approach analyzes a complex system, consisting for instance of of a wind farm and its siting and environmental conditions, by subdividing it in subsystems and unit problems to form a hierarchy of test cases with a systematic increase of complexity (Fig. 2) ([AIA98])
The building-block approach allows isolating individual or combined elements of the system, to segregate relevant physical phenomena in a more controlled setting that can be characterized more easily, and evaluate the predictive capacity of a model and estimate the potential impact of those elements on the full system performance. The process typically implies analyzing idealized conditions using theoretical approaches like similarity theory, parametric testing in a controlled environment with scaled-down models in wind tunnels and field testing of scaled or full-scale prototypes in research conditions as well as operational units in operational conditions. This hierarchy of increasing physical complexity is typically associated with decreasing levels of accuracy, in terms of data quality and resolution, because of practical as well as economical limitations. As mentioned previously, the validation space will always be limited to a limited range of system configurations and flow cases. The ultimate step in the building-block approach requires testing the model in operational conditions, where all phenomena are integrated. Here, the model can be calibrated and, eventually, fine-tuned to improve its predictive capacity (reduce bias and uncertainty).
The evaluation process can be considered an intrinsic part of technology innovation, i.e. translating ideas into added value of a product or service to meet specific needs. The innovation process originates from understanding the market needs, following a top-down or market-pull approach (Fig. 2) to define challenges that technology should solve. Alternatively, bottom-up or science-push innovation will use new knowledge to improve the “state-of-the-art” that feeds into the technology. In practice, both coexist although the market-pull approach should be the main driver to set expectations and avoid anchoring to knowledge niches.
In wind assessment applications, the product shall be a design tool whose core technology is a computational model. Innovation implies improving the predictive capacity of the model through better physical insight. Then, we use the model evaluation process to design experiments and validation cases that will allow us to test if certain model capabilities work as expected according to our conceptual model (our idea) and, more importantly, if this is actually adding value to the design tool.
This dual organization of the V&V process, in terms of interconnected exploration and exploitation cycles, can be described as an ambidextrous V&V process, in analogy with the term ambidextrous organization that would relate research and operational activities in the innovation process (O’Reilly and Tushman, 2004).
Ambidextrous model evaluation process implemented in the NEWA project (Sanz Rodrigo, 2019)¶
Fig. 3 illustrates this process in the context of the NEWA challenge of producing wind resource assessment methodologies based on a mesoscale-to-microscale model chain (Sanz Rodrigo, 2019). The challenge leads to formulating a concept for the model-chain through scientific review (Sanz Rodrigo, 2016c) and devising experiments to target all the relevant phenomena that should be captured. A validation hierarchy is defined to address these phenomena in a systematic way of increasing complexity (Sanz Rodrigo et al, 2016b). For example, Fig. 3 shows how the GABLS3 benchmark was used to demonstrate meso-micro coupling methodologies in the simulation of ABL flow along a diurnal cycle in flat terrain conditions. This case was used to implement the “tendencies” approach in microscale CFD models, which was then tested in operational conditions by integrating the model over one year at the Cabauw site to quantify performance in terms of relevant quantities of interest for wind resource assessment such as annual energy prediction (AEP). This model evaluation cycle is repeated as many times as possible to progressively incorporate additional phenomena from experimental campaigns and improve the physical insight of the model, at the right-hand side of the cycle, and long-term operational campaigns at the left-hand side to improve the statistical significance of the model in the application space.
Under the umbrella of international research networks like those promoted by IEA-Wind TCP research Tasks, it becomes natural to use the opportunity to coordinate large-scale experiments and validation programmes that would otherwise happen in a fragmented way. In order to implement an international model evaluation strategy it is necessary to count with a planning process that sets priorities along a unified validation directed research program. The planning process is shown in the top panel of Fig. 4, reprinted from Hills et al. (2015). It is composed of four phases:
Identify the objectives of the model from the perspective of the intended use (application) in terms of quantities of interest and the impact on the application.
Identify the phenomena of interest that the model should capture and prioritize the assessment based on the expected impact on the objectives.
Define a validation hierarchy that will allow to assess model performance for the prioritized phenomena.
Plan experiments to generate data for the validation hierarchy based on how the limited resources can be used most effectively.
The lower panel of Fig. 4 shows the process of experiment design, execution and validation activities that lead to the model assessment. The credibility step in the end determines, by expert judgment, to what extent the verification and validation results will improve the predictive capacity in the operational conditions of the model.
Validated directed program planning and execution (from Hills et al., 2015).¶
The implmenetation of the integrated program planning process for model validation in the Atmosphere to electrions (A2e) program can be found in Maniaci and Naughton ([MN19]).
An integrated program planning shall determine the links between knowledge gaps, experiment and model development needs and expected impact. The Phenomena Identification and Ranking Table (PIRT) is used as planning instrument to facilitate the collection and aggregation of information that is required to define and prioritize particular experimental validation activities (Pitch et al., 2001; Hills et al., 2015). This instrument relates the modeling requirements of the target application with the validation activities. By expert elicitation, it prioritizes experimental and validation tasks, following the building-block approach, to progressively and systematically build confidence on the models. The PIRT process is already established in the A2e programme with focus on wind farm models (Maniaci et al., 2017). It has also been adopted in the NEWA project for mesoscale to microscale atmospheric flow models (Sanz Rodrigo et al., 2016b).
Based on the needs of the application of interest and the associated modeling scope (Section 2), improving credibility is a matter of systematically addressing the phenomena of interest that are relevant for the model-chain to meet those needs. Hence, a PIRT is built to:
rank these physical and other related phenomena for the intended use;
characterize the adequacy of the model-chain, and the exiting experimental and validation datasets; and
perform gap analysis to identity the issues associated to the modeling of these phenomena and how they can be addressed.
Through expert elicitation, it is determined if a model has sufficient evidence to be used for the intended application and, if not, how to efficiently prioritize phenomena of interest that are expected to maximally improve model credibility within the available resources.
Table 1 shows different categories of phenomena of interest that could be included in the PIRT table through gap analysis. The phenomena are described in terms of associated issues (what the problem is) and the potential responses, i.e. what actions need to be taken to mitigate these issues. Examples of PIRT tables for wind energy can be found in Maniaci and Naughton (2017) and Sanz Rodrigo et al. (2016b).
Types of phenomena in a PIRT (adapted from Hills et al., 2015)¶
Type
Issues
Potential Responses
Physics
Important physics inadequately represented or missing
Model development or experimental characterization to better represent the phenomena; Model validation to assess the uncertainty associated with the lack of physics
Not clear if important phenomena, or interactions between phenomena, are adequately represented by model
Model validation to incorporate the effect of the phenomena
Ranking of phenomena not clear
Sensitivity analysis to rank importance for the quantities of interest
Model and Geometric Fidelity
Sub-components poorly represented
Sensitivity analysis of subsystem level with higher fidelity model to assess impact of underrepresented components
Geometric fidelity and/or grid resolution insufficient to capture behavior
Sensitivity analysis of subsystem level with higher fidelity model to assess impact of under-resolved geometry; Grid studies (solution verification) to characterize uncertainty due to grid dependencies
Fig. 5 provides a description of the high-level building-blocks established in the Wakebench framework. The V&V hierarchy addresses a two-sided multi-scale system consisting of the interplay between atmospheric scales (“wind”) and wind energy system scales (“wakes”). Atmospheric scales (dark grey blocks) range from surface-layer MOST conditions close to the ground, modified by terrain and vegetation, to turbulence across the ABL driven by mesoscale processes modulated by the regional wind climate. On the other side (light grey blocks) a wind energy system can range from a single turbine, a wind farm, a cluster of wind farms and, ultimately, the power system they are interconnected to.
Each scale has a number of physical phenomena, some of them listed in Fig. 5, which will be the basis of the PIRT process. As a whole, an integrated multi-scale model-chain for wind farm modeling will consist of inputs from the three blocks at the vertices of the triangle (turbine specifications, characterization of terrain and land-cover and initial and boundary conditions for the flow based on meteorological data from a global climate model (for instance, reanalysis data). The inner hexagon in the triangle defines the two-way couplings in the model-chain between the sub-system components. Depending on the application of interest, each of these sub-system models will have different fidelity levels. Each sub-system has its own V&V hierarchy down to unitary problem level as described in Fig. 2. The PIRT process will identify the shortcomings of each building-block and define V&V benchmarks to solve them.
System scales and phenomena of interest for “wind” (right) and “wake” conditions (left).¶
Integrated Experiment, Model Planning and Execution¶
The inegrated experiment, model planning and execution phase has many componenets that must interact between various program components, as outlined in the workflow shown in figure Fig. 6.
V&V Workflow, showing detailed components of the validation focused program.¶
AIAA. Guide: guide for the verification and validation of computational fluid dynamics simulations (AIAA g-077-1998(2002)). In Guide: Guide for the Verification and Validation of Computational Fluid Dynamics Simulations (AIAA G-077-1998(2002)). American Institute of Aeronautics and Astronautics, Inc., jan 1998. doi:10.2514/4.472855.001.
Richard G. Hills, David C. Maniaci, and Jonathan W. Naughton. V&v framework. Technical Report SAND2015-7455, SANDIA National Laboratories, sep 2015. URL: https://www.osti.gov/biblio/1214246-framework (visited on 2019-08-01).
Heather A. Holmes, Javier Sanz Rodrigo, Daniel Cabezón, and Michael Schatzmann. Model evaluation methodology for wind resource assessment. In WAUDIT Book of Proceedings, pages 82–112. WAUDIT Marie Curie ITN, 2013.
MMB+20
David C. Maniaci, Patrick J. Moriarty, Mathew F. Barone, Mathew J. Churchfield, Michael A. Sprague, and Srinivasan Arunajatesan. Wind energy high-fidelity model verification and validation roadmap. Technical Report SAND2020-1332, SANDIA National Laboratories, May 2020. URL: https://www.osti.gov/biblio/1634281 (visited on 2021-02-08), doi:10.2172/1634281.
David C. Maniaci and Jonathan W. Naughton. V&v integrated program planning for wind plant performance. Technical Report SAND2019-6888, SANDIA National Laboratories, June 2019. URL: https://www.osti.gov/biblio/1762662 (visited on 2019-08-01).
William L Oberkampf, Timothy G Trucano, and Charles Hirsch. Verification, validation, and predictive capability in computational engineering and physics. Applied Mechanics Reviews, 57(5):345, 2004. doi:10.1115/1.1767847.
William L. Oberkampf and Matthew F. Barone. Measures of agreement between computation and experiment: validation metrics. Journal of Computational Physics, 217(1):5–36, sep 2006. doi:10.1016/j.jcp.2006.03.037.
William L. Oberkampf and Timothy G. Trucano. Verification and validation in computational fluid dynamics. Progress in Aerospace Sciences, 38(3):209–272, apr 2002. doi:10.1016/s0376-0421(02)00005-2.
WEMEP is hosted in GitHub to become an open-source community project. For now, only invited participants from the Wakebench community are contributing to complete the first release.
These are some basic steps to get started when you are ready to contribute.
Of course you need a GitHub account. It is through your username that the project tracks your contributions.
Wind conditions is a generic term to refer to atmospheric flow quantities that affect wind turbine and wind farm performance in terms of energy production and structural integrity. This is the context for the application of atmospheric flow models in activities such as wind resource and energy yield assessment, wind turbine site suitability and wind farm design, during the planning phase, and weather and wind power forecasting during the operational phase of the wind farm. The IEA-Wind TCP Task 31 Wakebench is focused on the planning phase while Task 36 is dealing with wind power forecasting. The model evaluation framework shall focus on the wind farm system, considering all the mesoscale-to-microscale weather and turbulence processes, which are relevant for inflow and wind farm wake propagation and interaction.
Assessment of Wind Resource, Energy Yield and Turbine Suitability¶
The pre-construction wind resource and energy yield assessment process aims at predicting the net energy output of a wind farm over its lifespan. Based on the net energy yield and its associated uncertainty, the developer can judge the financial viability of the project. This estimate is defined in terms of a distribution that determines the exceedence probability of a certain annual energy production (AEP) within a specified timeframe (e.g. 20 years) (Fig. 7). The process to arrive to these estimates is described in [M12][CSF16].
Besides energy yield, the process also predicts mean and extreme wind conditions that are relevant for wind turbine design, i.e. to guarantee that siting conditions meet the requirements to ensure the structural integrity of the wind turbines as per the IEC 61400-1 [ITC8819a] and IEC61400-3 [ITC8819b] (fixed offshore) standards.
The IEC 61400-15 working group has recently defined a framework for a standardized reporting of the wind resource, energy yield and turbine suitability process:
The IEC 61400-15-1 complements IEC 61400-1 and 61400-3 in the reporting of site specific wind conditions and related atmospheric variables.
The IEC 61400-15-2 addresses the asessment and reporting of wind resource and energy yield.
Whenever possible we shall use the definitions provided therein on relevant quantities of interest for flow model evaluation. The variables are integrated with a wind speed distribution that is representative of the design lifetime and they are defined at hub-height (\(z_{hub}\)) unless otherwise stated.
Note
As of June 2020, the IEC 61400-15 standards are in draft form.
Annual average wind speed at hub height (\(V_{ave}\)): wind speed averaged according to the definition of annual average, i.e. mean value of a set of measured data of sufficient size and duration to serve as an estimate of the expected value of the quantity. The averaging time interval shall be a whole number of years to average out non-stationary effects such as seasonality.
Annual wind speed frequency distribution (\(f_{i,j}\)): Annual distribution of wind speeds as a function of wind direction i and/or wind speeds j. Wind speed is classified using 1 m/s bins and wind direction sectors are no wider than \(30^{\circ}\). Additional dimmensions related to turbulence characteristics like stability, turbulence intensity or wind shear could be used for additional granularity in the distribution.
Weibull distribution: The probability distribution function used to describe the distribution of wind speeds over a period of one year, defined in terms of the scale parameter (\(C\)) and shape parameter \(k\).
Gross annual energy production (\(AEP_{gross}\)): total amount of electrical energy produced by the Wind Turbine Generator System (WTGS), estimated by integrating the power curve with the wind speed frequency distribution and multiplying by the number of hours in a year. For a wind farm:
where \(f_{i,j,k}\) is the annual wind speed frequency distribution at each turbine site k, \(P_k(V_j)\) is the power curve of each turbine at wind speed \(V_j\) and \(T\) = 8760 h is the number of hours in a year. The gross AEP is also defined as the AEP at a reference site, typically a meteorological mast, vertically extrapolated to hub-height and horizontally extrapolated to the turbine sites. This extrapolation is carried out by profile methods and flow models that predict speed-up effects due to terrain elevation and roughness changes, forest canopies and obstacles.
(Net) Annual energy production (\(AEP\)): total amount of electrical energy delivered at the grid connection point after deducing all the energy losses that take place in the wind farm.
where \(\eta_{l}\) is the efficiency corresponding to the loss category l as per the IEC 61400-15, namely: electrical, availability, wake effect, curtailment, environmental and turbine performance. Each category includes a number of subcategories. For instance, wake losses are subdivided into internal (interarray effects), external (current farm-farm effects) and future (prospect farm-farm effects) wake losses. Internal wake losses are predicted by wake models to obtain the, so-called, array efficiency:
where the efficiency is defined in terms of a power ratio with \(Pw_k\) being the power output predicted by the wake model at each turbine position k.
Annual capacity factor (\(CF\)): the ratio between the AEP and the maximum possible annual energy output, an ideal case where all turbines would be producing at rated power throughout the year.
\[CF = \frac{AEP}{T\sum_{k} Prated_k}\]
where \(Prated_k\) is the rated power of each turbine. Alternatively, wind farm performance is defined in terms of the annual equivalent hours of the wind farm operating at rated power, i.e. \(AEH = CF \cdot T\)
The pre-construction energy yield assessment process will output a distribution of AEP defined in terms of the median \(P50\), where the actual AEP would be exceeded 50% of the time, and a standard deviation \(\sigma_{AEP}\) as a measure of the AEP uncertainty. Then, the prediction bias is the difference between the estimated P50 and the actual AEP
\[BIAS_{AEP} = AEP_{true} - AEP_{P50}\]
While each quantity of interest can be subject to uncertainty quantification individually, the main focus of the IEC 61400-15-2 standard is to predict the overall energy production uncertainty since this is directly connected to the financial performance of a wind project. This overall uncertainty is broken down into categories and subcategories by the standard to provide a common framework for the wind industry. Lee and Fields (2020) [LF20] provide a review of energy yield assessment prediction bias, losses and uncertainties following this framework. The review shows that while there has been a tendency towards the overestimation of P50, this has been progressively corrected and we are now approaching zero bias on average. The estimated mean AEP uncertainty remains at over 6% implying that there is room for improvement. Indeed, changing the uncertainty by 1% can lead to 3-5% change in the net present value of a wind farm [LF20].
Extreme wind speed with a recurrence interval of 50 years (\(V_{50}\)): Also called reference wind speed (\(V_{ref}\)) in the IEC 61400-1 standard to define WTGS classes. A turbine designed for a WTGS class with a reference wind speed \(V_{ref}\), is designed to withstand climates for which the extreme 10 min average wind speed with a recurrence period of 50 years at turbine hub-height is lower than or equal to \(V_{ref}\).
Annual average flow inclination angle (\(\phi\)): The flow inclination is defined as the angle between a horizontal plane and the wind velocity vector at hub height:
\[\phi = tan^{-1}\left(V_z/V_{xy}\right)\]
where \(V_{xy}\) and \(V_{z}\) are the horizontal and vertical components of the wind speed. The flow inclination angle is positive if the wind velocity vector is pointing upwards. The annual average shall be taken as the energy weighted mean from all directions.
Mean turbulence intensity (\(I\)): the ratio of the wind speed standard deviation to the mean wind speed determined from the same set of measured wind speed data and taken over a period of 10 minutes and a minimum sampling frequency of 5 seconds.
\[I = \frac{\sigma_V}{V}\]
Standard deviation of turbulence intensity (\(\sigma_{I}\)): The standard deviation of a sub set of the turbulence intensities (\(I\)). The sub set typically represents a bin within a wind speed- wind direction matrix.
Average turbulence intensity at 15m/s (\(I_{15}\)): Mean turbulence intensity over all wind directions in 15m/s wind speed bin.
Extreme ambient turbulence intensity: Extreme value of the ambient turbulence intensity with a return period of 50 years as a function of wind speed.
Mean wind shear (\(\alpha\)): Wind shear, i.e. the variation of wind speed across a plane perpendicular to the wind direction, (or power law) exponent.
Sanz Rodrigo et al. (2016) [RAM+17] provide a review of mesoscale-to-microscale wind farm flow models of different fidelity levels considering meteorological and wind energy terminology. Each scale has different applications and quantities of interest, which will determine the orientation of the model evaluation strategy (Fig. 8).
Models can be coupled together to form a multi-scale modeling system where, for instance, the microscale sub-system (the wind farm) uses input data generated by the mesoscale sub-system to characterize the long-term wind climate distribution that modulates the local wind conditions. Similarly, an aerolastic model of the turbine sub-system can be used to predict detailed rotor aerodynamics responsible for wake generation in the wind farm wake model. A mind map elaborated during IEA-Task 31 Phase 3 shows the relationships between model building-blocks at different scales, input quantities and phenomena of interest for the intended use of these models (Fig. 9).
Mind map of multi-scale models and phenomena of interest for wind conditions.
Interactive mind map¶
The mind map breaks down the full complexity of atmospheric models into three scales:
Global: drives wind climate variability from seasons to decades at horizontal scales of tens of kilometers. Global reanalyses are typically used to characterize this variability and serve as input boundary condtions for mesoscale models.
Mesoscale: drives weather processes at regional level down to scales of the order of 1 km. Relevant mesoscale phenomena include: horizontal wind speed (gross AEP) gradients due large-scale topography, land-sea transitions and farm-farm (external wake) effect, low-level jets producing large wind shear during stable conditions, etc. Mesoscale models provide forcing for microscale models in the form of virtual masts, generalized wind climates, lateral boundary conditions or volumetric forzes (also called tendencies).
Microscale: drives turbulence and speed-up effects at site level at scales down to a few meters. Site effects depend on local changes in elevation and roughness, the presence of obstacles and forest canopies as well as thermal stratification across the atmospheric boundary-layer (ABL). Microscale effects are particularly important in complex terrain where relevant phenomena develop such as: flow separation and recirculation, gravity waves, gap flow, hydraulic jump, mountain-valley winds, etc. At microscale, wind farm wake models are embedded in atmospheric flow models to simulate internal wake effects that determine array efficiency.
Wake models are described in detail in the Multi-Scale Wind Farm Modeling section to simulate external and internal wake effects (array efficiency) and wind turbine loads.
Following the model evaluation process of Fig. 3, a validation strategy for wind conditions requires the provision of high-fidelity experiments targeting high-impact phenomena of interest, to validate the capacity of the flow model to deal with relevant physics, as well as long-term wind resource campaigns to demonstrate the added value that resolving these phenomena brings to the applications of interest. These applications typically involve integrating a discrete number of flow model simulations with a statistical methodology that provides the expected long-term mean or extreme values and the associated uncertainties. This should be done for a wide variety of wind climates and siting conditions to cover the widest operational range possible.
This validation strategy was implemented in the New European Wind Atlas (NEWA) project towards the development of a new methodology for the assessment of wind conditions that is based on a mesoscale to microscale model-chain approach [RAW+20]. The scope of the project was focused on wind conditions for wind resource and site assessment, i.e. without the influence of wind turbines. Therefore, the model-chain was devoted to atmospheric flow models with two applications in mind:
Wind atlas for regional planning: the main focus was on the mesoscale model, to come up with a reference set-up of the Weather Research and Forecasting (WRF) model that could be used seamlessly across Europe. Through sensitivity analysis, the most suitable configuration was selected to produce a 30-year long simulation forced by ERA5 reanalysis data [HSW+20]. Then, this long-term wind climate was statistically downscaled to 50 m using the WAsP methodology. Hence, the wind atlas model-chain consists on physical downscaling down to 3 km resolution, to produce long-term time series of mesoscale wind characteristics, whose long-term wind climate distributions are then used as input data for a microscale model to produce high-resolution wind resource quantities. The validation strategy was based on a database of 291 meteorological masts, at least 40 m tall, made available by Vestas [DOW+20]. The main objective of the validation campaign was to determine the general quality of the wind atlas, categorized by regions and terrain complexity, determined by the ruggedness index RIX (Fig. 10) [MTL08]. A one year multi-physics ensemble run was also used to quantify the spread of mesoscale winds which would translate into input uncertainty for microscale models [RBH+19].
Site assessment: here the main focus was on the microscale model, in particular, in the implementation of mesoscale forcing and boundary conditions for heterogeneous topography (complex terrain and forest canopies) using both Reynolds-Averaged Navier Stokes (RANS) and Large-Eddy Simulation (LES) turbulence models. Hence, the validation strategy seeked validation cases from detailed experiments where these modeling feutures would be tested in the prediction of mean flow and turbulence quantities. The range of experiments carried out in the NEWA allows testing over a wide range of siting conditions from offshore to coastal transitions to smooth and complex terrain with a without forest canopies. Next section provides an overview of these experiments and other open-access datasets that can be used to validate flow models.
The building-block validation hierarchy of Fig. 11 provides a framework to map validation datasets with phenomena of interest at different scales and show how they complement each other to cover a reasonably wide range of wind conditions. Site effects are modulated by the regional wind climate which is driven by synoptic weather and local mesoscale processes.
Building-block validation hierarchy and phenomena of interest for wind conditions.¶
While long-term statistics of wind conditions are traditionally characterized with meteorological masts, many recent experiments include an intense operational period (IOP) of several weeks to months with extensive use of remote sensing equipment. In particular, scanning lidar systems allow to characterize the spatio-temporal structure of the flow field along scanning trajectories following a particular transect, or slicing vertical or horizontal planes ([MAA+17]). Table 2 provides a summary of experiments and other sources of open-access data for the validation of atmospheric flow models.
Summary of open-access datasets for the validation of flow models for wind conditions¶
Dataset
Location
Period
Site Conditions
Key References & Repositories
Homogeneous ABL
Cabauw
This 200-m tall mast has been a reference in ABL research for its horizontally homogeneous conditions.
The GABLS3 flow case is a diurnal cycle developing a strong nocturnal low-level [BBvM+14].
116 m high smooth hill isolated in all wind directions but the NE-E sector. Over 50 masts installed, 35 of them of 10 m
height installed along transects following the main axes of the hill
200-m tall forested hill equipped with a 200-m mast at the hill top, a 140-m mast at the inflow and scanning Doppler lidars
mapping a transect along the prevailing wind direction
10-km long transect consisting of 9 remote sensing profilers and one 180-m flux-profile mast in forested and moderately
complex terrain. Surface pressure gradient measurements
12-m high hill surrounded by water in all directions except to the E. An almost vertical escarpment faces the prevailing W-SW
sector. 10 masts equiped with sonic and cup anemometers.
5 scanning Doppler lidars measuring a Z-shaped 10-km long transect along the ridge tops and across the valley together
with a windRASS profiler, 7 tall masts and 10 surface stations
Following the building-block approach illustrated in Fig. 11, we provide a comprehensive list of benchmarks for the design and testing of atmospheric flow models. These test cases are derived from theory and high-fidelity experiments to target specific flow phenomena that is considered relevant for the intended uses of the models. From Monin-Obukhov theory in homogeneous conditions to flow over complex terrain and forest canopies, a hierarchy of flow cases is provided in each building-block.
The ABL building-block of Fig. 12 deals with the horizontally homogeneous atmospheric boundary-layer. A hierarchy of verification and validation cases is suggested to progressively incorporate essential physics, namely:
Monin-Obukhov similarity theory (MOST) for surface-layer steady-state conditions depending on roughness and stability [MO54].
LeipzigABL wind profile in neutral steady-state conditions [Let50].
GABLS1stable ABL under uniform geostrophic wind and surface cooling [CHB+06].
GABLS2diurnal cycle under uniform geostrophic wind and varying surface temperature [KSH+10].
GABLS3 diurnal cycle under realistic mesoscale forcing and varying surface boundary conditions [BBvM+14].
Benchmarks for MOST and Leipzig were conducted during Wakebench Phase 1 [RGA+14]. The GEWEX Atmospheric Boundary Layer Studies (GABLS), developed by the boundary-layer meteorology community [HSB+13], can be adopted by the wind energy community by focusing the evaluation on rotor-based quantities of interest [SRCK17]. The GABLS3 and Cabauw benchmarks were conducted in Wakebench Phase 2 [RAA+17][RAG+18].
Participants show good match with MOST at rotor heights (Fig. 13). Next to the wall some deviations occur in one \(k-\epsilon\) model due to a too large first-cell height. The LES model is not designed to run under surface layer conditions and show decreasing tke with height typical of ABL models.
This verification benchmark is intended for anyone using surface layer models as a precursor to any other validation case since it enables fundamental physics in flat terrain conditions. Simulations of homogeneous profiles in an empty domain is also performed to verify the equilibrium of the wall functions with the turbulent flow model [BSC07].
The objectives of the benchmark are:
Demonstrate that the flow model, when running in MOST conditions, is able to reproduce the analytical profiles of the theory in neutral and stratified conditions.
Verify that wall boundary conditions are in equilibrium with the turbulence model for a range of surface roughness conditions.
Monin Obukhov similarity theory (MOST) [MO54] sets the point of departure of modern micrometeorology [Fok06]. It is valid in the surface layer, i.e. approximately in the first 10% of the ABL, where Coriolis effects are negligible compared to friction, and under stationary and horizontally homogeneous conditions with no radiation. In these ideal conditions the vertical variations of wind direction, shear stress, heat and moisture fluxes are constant. MOST states that any dimensionless turbulence characteristic will only depend on a reduced set of scales. In addition to friction velocity (\(u_*\)) and the height above the ground (\(z\)), as basic scales in neutral conditions, the surface (virtual) potential temperature \(\Theta_0\) and kinematic heat flux (\(\overline{\omega\theta}\)) are also required in thermally stratified conditions. The Obukhov length scale \(L\) is made of a combination of these parameters,
where \(g\) is the gravity and \(\kappa\) is the von Karman constant. A dimensionless height \(\zeta = z/L\) is used as stability parameter (\(\zeta < 0\) for unstable, \(\zeta > 0\) for stable and \(\zeta = 0\) for neutral conditions). Any dimensionless turbulent characteristic will depend solely on this parameter. By integration of the velocity and potential temperature gradients, well-known logarithmic profiles are obtained:
where \(U\) is the mean velocity at height \(z\), \(z_0\) and \(z_{0t}\) are the roughness length for momentum and heat, \(k\) is the turbulent kinetic energy, \(\epsilon\) is the turbulent dissipation rate, \(\Theta_*\) is a temperature scale,:math:Theta is the mean (virtual) potential temperature at height \(z\), \(C_{mu}\) is a constant, and \(\phi_x\) and \(\psi_x\) are stability functions obtained from flux-profile experiments in flat terrain, e.g. [PD84]:
MOST is used to design wind engineering surface layer models. When an empty domain is simulated in steady-state and homogeneous surface conditions the flow should produce the fully-developed log-profiles predicted by the theory. These are the conditions that will be simulated in this test case.
This verification test was followed by Richards and Hoxey (1983) [RH93], who calibrated the RANS \(k-\epsilon\) turbulence model by enforcing consistency with MOST in the surface layer in neutral conditions. Similarly, Alinot and Masson (2005) [AM05] followed the same approach to derive consistency conditions for a \(k-\epsilon\) model in stratified conditions.
Please use dry air with a density \(\rho = 1.225 kg m^{-3}\) and dynamic viscosity \(\mu = 1.73e-5 kg m^{-1} s^{-1}\). The von Karman constant is \(\kappa = 0.4\).
Input profiles can be found in this data repository: [Rod12]
An empty domain of 3x0.5x0.5 km (x,y,z) dimensions should be simulated with three different values of roughness length in the ground wall.
Neutral case:
Run 1: \(z0 = 0.0002 m\), typical of offshore conditions
Run 2: \(z0 = 0.03 m\), typical of open fields with low vegetation
Run 3: \(z0 = 0.4 m\), typical of forested terrain
Stratified case:
Run 1: \(L = -100 m\), for unstable conditions equivalent to a kinematic heat flux of \(\overline{\omega\theta} = 0.047 mKs^{-1}\)
Run 2: \(L = \infty\), for neutral conditions
Run 3: \(L = 100 m\), for stable conditions equivalent to a kinematic heat flux of \(\overline{\omega\theta} = 0.047 mKs^{-1}\)
The origin of the coordinate system will be placed in the middle of the bottom edge of the inlet wall. The modeler is free to configure the computational grid according to own criteria.
Please provide output vertical profiles of mean velocity \(U\), potential temperature \(\Theta\) and turbulent kinetic energy \(k\) at the outlet (x = 3000 m, y = 0) using the file naming and format convention described in the Windbench user’s guide with profID = outlet# (# is the run number = [1,2,3]). Mention the friction velocity \(u_*\) used in the study since this input will be used to normalize the results.
Please describe in detail the way stratification is handled by the model in terms of stability functions, boundary conditions, contributions to turbulence equations, etc.
A S Monin and A M Obukhov. Basic laws of turbulent mixing in the surface layer of the atmosphere. Contrib. Geophys. Inst. Acad. Sci. USSR, pages 163–187, 1954.
Javier Sanz Rodrigo, Pawel Gancarski, Roberto Chavez Arroyo, Patrick Moriarty, Matthew Chuchfield, Jonathan W. Naughton, Kurt S. Hansen, Ewan Machefaux, Tilman Koblitz, Eoghan Maguire, Francesco Castellani, Ludovico Terzi, Simon-Philippe Breton, Yuko Ueda, John Prospathopoulos, Gregory S. Oxley, Carlos Peralta, Xiadong Zhang, and Björn Witha. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. J. Phys.: Conf. Ser., 524:012105, June 2014. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F524%2F1%2F012105 (visited on 2020-10-14), doi:10.1088/1742-6596/524/1/012105.
Javier Sanz Rodrigo and Patrick Moriarty. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. June 2014. URL: https://zenodo.org/record/4088287 (visited on 2020-10-14), doi:10.5281/zenodo.4088287.
Fig. 16 shows two RANS and one LES simulations. Both RANS models use a length-scale limiter (\(\lambda \approx 42 m\)) to monotonic linear growth of the mixing legth with height that surface-layer models would predict. Both RANS simulations differ in the value of \(\lambda\) but produce results consistent with the Leipzig data. The LES results show too much mixing in the upper part of the boundary layer resulting in too deep boundary layer.
The case is suitable for the verification of ABL models in neutral conditions. The objective is to demonstrate how ABL models reproduce the characteristic Ekman spiral driven by uniform geostrophic forcing.
The measurements were done on a grass-covered airfield with flat surroundings. Upstream, the air passes over the city of Leipzig. The Leipzig wind profile results from a set of 28 pitot-balloon observations with two theodolites, between 9:15 and 16:15 on October 20, 1931, during stable weather [P32]. During the experiment, the surface isobars were rectilinear indicating that the geostrophic conditions were steady and the horizontal gradients were negligible.
Lettau (1950) [Let50], performed a reanalysis of the measurements which resulted in a smooth profile, a “representative average” of the original, more scattered data. This classical profile has been discussed extensively in the literature. The boundary layer meteorology folklore considers this profile as a reference for an idealized neutral, barotropic (geostrophic wind constant with height), horizontally homogenous steady-state atmospheric boundary layer (ABL). However, it has been also argued that the profile was obtained in slightly stable conditions with an Obukhov length in the order of 500 m obtained by profile fitting in the lower 150 m [Sun79]. In fact, Lettau (1950) reports a lapse rate of potential temperature of 0.35 K / 100 m.
The Leipzig wind profile has been extensively used for the analysis and design of ABL models. Blackadar (1962) [Bla62] derived his well known analytical expression for the ABL mixing length profile in flat terrain making use of this profile. The limiting value of the mixing length was found to be proportional to the ratio of the geostrophic wind and the Coriolis parameter. He assumed that the slight stratification of the profile did not influence its turbulence structure. Many mixing-length models of the ABL are based on Blackadar’s parameterization ever since.
Detering and Etling (1985) [DE85] proposed a k-ε model of the ABL that could reduce the excessive mixing of the default turbulence model of Launder and Spalding (1974) [LS74]. A similar strategy was followed by Apsley and Castro (1997) [AC97] using a length-scale limiter to avoid the quasy-linear growth of the mixing length beyond the surface layer.
Riopellle and Stubley (1989) [RS89] used a second-order turbulence closure that included stable stratification and found better agreement with the Leipzig profile than if neutral conditions were assumed.
Even though it is quite old, the Leipzig profile is useful because of the steady barotropic conditions of the experiment. Being a well-established reference, it is suitable for verification and model intercomparison studies. However, since the dataset does not include thermal stratification properties, it should not be treated as a complete model validation dataset.
The validation data consists on vertical profiles of velocity components and eddy viscosity as estimated by Lettau (1950) [Let50]. They can be found in this data repository: [Rod12]
Please provide vertical profiles of velocity components (U,*V*), turbulent kinetic energy (tke) and turbulent viscosity (nu_t) using the file naming and format convention described in the Windbench user’s guide with profID = outlet. Hence, the output profile file contains the following variables (header), in this order: Z(m), U(m/s), V(m/s), tke(m2/s2), nu_t(m2/s).
This benchmark is based on prescribed boundary conditions in order to evaluate the scatter of different ABL models. You can try to guess the stability conditions by running a quasi-steady stratified case and uniform cooling.
DAVID D. APSLEY and IAN P. CASTRO. A limited-length-scale k-ε model for the neutral and stably-stratified atmospheric boundary layer. Boundary-Layer Meteorology, 83(1):75–98, April 1997. URL: https://doi.org/10.1023/A:1000252210512 (visited on 2020-10-15), doi:10.1023/A:1000252210512.
H. W. Detering and D. Etling. Application of the E-ε turbulence model to the atmospheric boundary layer. Boundary-Layer Meteorol, 33(2):113–133, October 1985. URL: https://doi.org/10.1007/BF00123386 (visited on 2020-10-15), doi:10.1007/BF00123386.
G. Riopelle and G. D. Stubley. The influence of atmospheric stability on the ‘Leipzig’ boundary-layer structure. Boundary-Layer Meteorol, 46(3):207–227, February 1989. URL: https://doi.org/10.1007/BF00120840 (visited on 2020-10-15), doi:10.1007/BF00120840.
Javier Sanz Rodrigo, Pawel Gancarski, Roberto Chavez Arroyo, Patrick Moriarty, Matthew Chuchfield, Jonathan W. Naughton, Kurt S. Hansen, Ewan Machefaux, Tilman Koblitz, Eoghan Maguire, Francesco Castellani, Ludovico Terzi, Simon-Philippe Breton, Yuko Ueda, John Prospathopoulos, Gregory S. Oxley, Carlos Peralta, Xiadong Zhang, and Björn Witha. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. J. Phys.: Conf. Ser., 524:012105, June 2014. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F524%2F1%2F012105 (visited on 2020-10-14), doi:10.1088/1742-6596/524/1/012105.
Javier Sanz Rodrigo and Patrick Moriarty. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. June 2014. URL: https://zenodo.org/record/4088287 (visited on 2020-10-14), doi:10.5281/zenodo.4088287.
A. Sundararajan. Some aspects of the structure of the stably stratified atmospheric boundary layer. Boundary-Layer Meteorol, 17(1):133–139, August 1979. URL: https://doi.org/10.1007/BF00121941 (visited on 2020-10-15), doi:10.1007/BF00121941.
Diurnal Cycle Leading to a Nocturnal Low-Level Jet (GABLS3)¶
The GABLS3 benchmark was organized within Wakebench Phase 2 with support from the NEWA and MesoWake EU projects. The results were presented at the Wake Conference 2017.
The main challenge for microscale models was to produce consistent flow fields with respect to the reference mesoscale model that was used to derive their input forcings. This consistency has been achieved by URANS and LES models. The spread of the models is significant but of similar magnitude to that shown by the ensemble of WRF simulations. Using the WRF ensemble mean leads to the best predictions of wind shear and wind veer. Fig. 17 shows the evolution of quantities of interest in the rotor area. The phase error in the input mesoscale data dominates the error at microscale that, nevertheless, are able to produce a realistic nocturnal low-level-jet.
A sensitivity analysis on input forcings and surface boudary conditions with CFDWind URANS single-column model was presented at Torque 2016 and in a follow-up Wind Energy Science paper.
The added value of introducing mesoscale forcings to a single-column model is investigated by systematically removing forcing terms in the momentum and heat equation. The largest effect comes from geostrophic wind which needs to be height-dependent to track wind conditions in the upper levels of the ABL. Here it also becomes important to introduce advection tendencies which can be quite intense during low-level jet conditions. The bias in the mesoscale input forcings can be mitigated by introducing nudgging terms (Fig. 18).
The GABLS3 case was selected in the NEWA project as a golden benchmark for the design of mesoscale-to-microscale methodologies for wind resource assessment. The case is suitable for the development of microscale wind farm models that incorporate realistic forcing, derived from a mesoscale model, along a typical diurnal case that leads to the development of a nocturnal low-level jet. GABLS1 and GABLS2 can be considered as precursor cases dealing with turbulence modelling of the atmospheric boundary layer under idealized forcing conditions. These cases are suitable for the design of RANS-based single-column models by comparison with LES simulations, that have shown high consistency [SHK+11][BBL+10][SRCK17].
Challenges of the GABLS3 case include: incorporating time- and height-dependent mesoscale forcing in microscale models, turbulence modeling at varying atmospheric stability conditions, defining suitable surface boundary conditions for momentum and heat and characterization of the wind profile in (non-logarithmic) LLJ conditions.
Wind-energy specific objectives of the benchmark include:
Demonstrate the capability of wind energy ABL models to incorporate realistic mesoscale forcing
Implement surface boundary conditions suitable for wind assessment studies using mesoscale simulation data and/or observations (typical of wind energy campaigns)
Develop suitable model calibration strategies for wind energy applications or, in other words, how to best use available measurements (typical of wind energy campaigns) to correct meso-micro predictions
Define suitable metrics for validation of ABL models based on wind energy quantities of interest
By “typical wind energy campaigns” we would like to encourage modellers to prioritize observations that are common place in wind resource assessment campaigns (80 masts with velocity and temperature measurements, lidar profilers measuring up to 400 m).
The GEWEX Atmospheric Boundary Layer Study (GABLS) series of benchmarks were developed by the boundary-layer meteorology community to improve the representation of the atmospheric boundary layer in regional and large-scale atmospheric models. The model intercomparison studies have been organized for single-column models (SCM) and large-eddy simulation models (LES). The cases are based on observations over flat terrain in the Artic, Kansas (USA), Cabauw (The Netherlands) and Dome C (Antarctica).
GABLS1 simulated a quasi-steady stable boundary layer resulting from 9 hours of uniform surface cooling [Hol06][CHB+06][SHK+11]. GABLS2 simulated a diurnal cycle, still with idealized forcing, by simplifying measurements from the CASES-99 experiment in Kansas [KSH+10][BBL+10]. GABLS3 simulated a real diurnal case with a strong nocturnal low-level jet (LLJ) at the Cabauw met tower in the Netherlands [Hol14][BBvM+14][BBS+14][BHB12][HSB+13]. In GABLS4, the aim is to study the interaction of a boundary layer with strong stability in relatively simple surface coupling characteristics.
The challenges of stable boundary layers and diurnal cycles are reviewed by Hotlslag et al. (2013) [HSB+13], notably: the relation between enhanced mixing in operational weather models performance, investigate the role of land-surface heterogeneity in the coupling with the atmosphere, develop LES models with interactive land-surface schemes, create a climatology of boundary-layer parameters (stability classes, boundary-layer depth, and surface fluxes) and develop parameterizations for the very stable boundary layer when turbulence is not the dominant driver. These challenges are ultimately shared by wind energy applications that are embeded in atmospheric models.
The GABLS3 set-up is described in Bosveld et al. (2014) [BBvM+14]. The case analyzes the period from 12:00 UTC 1 July to 12:00 UTC 2 July 2006, at the KNMI-Cabauw Experimental Site for Atmospheric Research (CESAR), located in the Netherlands (51.971ºN, 4.927ºE), with a distance of 50 km to the North Sea at the WNW direction [12]. The elevation of the site is approximately -0.7 m, surrounded by relatively flat terrain characterized by grassland, fields and some scattered tree lines and villages (Fig. 19). The mesoscale roughness length for the sector of interest (60º - 120º) is 15 cm.
Land-use map of a 30x30 km area around the Cabauw site (figure from KNMI’s HYDRA project website)¶
The CESAR measurements are carried out at a 200-m tower, free of obstacles up to a few hundred meters in all directions. The measurements include 10-min averaged vertical profiles of wind speed, wind direction, temperature and humidity at heights 10, 20, 40, 80, 140 and 200 m, as well as surface radiation and energy budgets. Turbulence fluxes are also monitored at four heights: 3, 60, 100 and 180 m. A RASS profiler measures wind speed, wind direction and virtual temperature above 200 m.
The selection criteria for GABLS3 consisted on the following filters applied to a database of 6 years (2001 - 2006): stationary synoptic conditions, clear skies (net longwave cooling > 30 W/m² at night), no fog, moderate geostrophic winds (5 to 19 m/s, with less than 3 m/s variation at night) and small thermal advective tendencies. Out of the 9 diurnal cycles resulting from this filtering process, the one that seemed more suitable was finally selected: 12:00 UTC 1 July to 12:00 UTC 2 July 2006.
More information about the case background and set-up can be found in the official GABLS3 website
The case set-up and input data of the original GABLS3 case can be found in the KNMI website. This is usefull if you want to compare with published results of the original SCM model intercomparison. In the original GABLS3 set-up, the simulated mesoscale tendencies are adjusted to produce a better match with the surface geostrophic wind obtained from a network of synoptic stations and the wind speed at 200-m measured at the Cabauw tower. Initial profiles are based on soundings measured near the Cabauw mast.
Alternatively, you can use inputs generated entirely from a WRF simulation, as described in [RCK16b][SRCK17]. Here, instead of using observed initial projiles and adjusted mesoscale forcings you can use initial profiles and forcing produced directly from a mesoscale simulation. This is more representative of a wind energy model-chain set-up, where the inputs of a microscale model are generated by a “wind atlas” methodology that doesn’t normally include corrections with local measurements. Instead, local adjustments are allowed at the microscale level by incorporating onsite measurements as if these measurements were part of a typical wind resource assessment campaign.
The WRF simulation is based on a one-way nesting configuration of three concentric square domains centered at the Cabauw site, of the same size 181x181, and at 9, 3 and 1 km horizontal resolution. The vertical grid, approximately 13 km high, is based on 46 terrain-following (eta) levels with 24 levels in the first 1000 m, the first level at approximately 13 m, a uniform spacing of 25 m over the first 300 m and then stretched to a uniform resolution of 600 m in the upper part. The U.S. Geological Survey (USGS) land-use surface data, that comes by default with the WRF model, is used together with the unified Noah land-surface model to define the boundary conditions at the surface. Other physical parameterizations used are: the rapid radiative transfer model (RRTM), the Dudhia radiation scheme and the Yonsei University (YSU) first-order PBL scheme. The simulation uses input data from ERA-Interim with a spin-up time of 24 hours. The WRF set-up follows the reference configuration of Kleczek et al (2014) [KSH14], who run a sensitivity analysis of WRF showing reasonably good results at reproducing the nocturnal LLJ.
A NetCDF file is provided with the following information:
Site coordinates and Coriolis parameter
Time-height 2D arrays of velocity components (U,V,W) and potential temperature (Th)
Time-height 2D arrays of mesoscale forcings (tendencies): geostrophic wind (Ug, Vg), advective wind (Uadv, Vadv) and advective potential temperature (Thadv)
Time array of surface-layer quantities: friction velocity (ust), kinematic heat flux (wt), 2-m temperature (T2), skin temperature (TSK), surface pressure (Psfc)
Units, dimmensions and variables description are all provided in the NetCDF file. Momentum tendencies (Fig. 20) should be multiplied by the Coriolis parameter to obtain appropriate forces in [m/s]. For convenience, we have ommitted information about humidity since the assumption of dry-atmosphere is typically adopted by wind energy flow models.
Time-height contour plots of the longitudinal wind component U and momentum budget terms: Utend = Uadv + Ucor + Upg + Upbl. [1][2]¶
A python script is provided to show how to read the NetCDF input file and extract these variables.
The following quantities of interest (QoI) will be evaluated as described in [RCK16b][SRCK17], using a reference rotor size of 160 m diameter at a hub-height of 120 m (~ 7 MW turbine):
Rotor equivalent wind speed (REWS)
Hub-height wind direction (WDhub)
Turbulence intensity at hub-height (TIhub)
Wind shear (power-law exponent α) and wind veer (slope of linear fit to wind direction differences ψ) accross the rotor plane
Surface-layer quantitites: T2, ust, wt and z/L
The evaluation consists on time-series plots of these QoIs along the diurnal cycle and mean-absolute error (MAE) integrated over the whole cycle.
The benchmark is mainly developed for microscale models that make use of the input data described above. However, mesoscale or multi-scale (online meso-micro) simulations are also welcome. The following suggestions are provided to guide the model runs:
We shall use the 2-m temperature (T2) as our most practical reference to deduce the potential temperature surface boundary conditions using Monin Obukhov similarity theory, since this variable is routinely measured in measurement campaigns and is part of the standard output of meteorological models.
Simulations may be based entirely on the mesoscale input data or incorporate measurements from the Cabauw mast. Priority should be given to measurements that can be found in “typical wind energy campaigns” (80 masts with velocity and temperature measurements, lidar profilers measuring up to 400 m).
Sensitivity analysis of mesoscale models can be used to quantify the input uncertainty derived from the spread of the ensemble of simulations.
Online multi-scale simulations models can be used as a reference for microscale models that are coupled to mesoscale asyncronously through the input data. To allow this comparison multi-scale simulations should be also run with ERA-Interim input data.
Microscale models using Sogachev et al. (2012) [SKL12]\(k-\epsilon\) turbulence model shall use this set of constants: \(κ = 0.4\), \(C_{ε1} = 1.52\), \(C_{ε2} = 1.833\), \(σ_k = 2.95\), \(σ_ε = 2.95\) and \(C_μ = 0.03\)
If resources allow, please use a spin-up time of 24 hours as in the input data.
P. Baas, F. C. Bosveld, G. Lenderink, E. van Meijgaard, and A. a. M. Holtslag. How to design single-column model experiments for comparison with observed nocturnal low-level jets. Quarterly Journal of the Royal Meteorological Society, 136(648):671–684, 2010. _eprint: https://rmets.onlinelibrary.wiley.com/doi/pdf/10.1002/qj.592. URL: https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.592 (visited on 2020-10-15), doi:10.1002/qj.592.
S. Basu, A. a. M. Holtslag, and F. C. Bosveld. GABLS3-LES Intercomparison Study. In Proceedings of the Workshop on Diurnal cycles and the stable boundary layer, 7-10 November 2011, Reading, UK, 75–82. European Centre for Medium-Range Weather Forecasts, 2012. URL: https://research.wur.nl/en/publications/gabls3-les-intercomparison-study (visited on 2020-10-15).
Fred C. Bosveld, Peter Baas, Gert-Jan Steeneveld, Albert A. M. Holtslag, Wayne M. Angevine, Eric Bazile, Evert I. F. de Bruijn, Daniel Deacu, John M. Edwards, Michael Ek, Vincent E. Larson, Jonathan E. Pleim, Matthias Raschendorfer, and Gunilla Svensson. The Third GABLS Intercomparison Case for Evaluation Studies of Boundary-Layer Models. Part B: Results and Process Understanding. Boundary-Layer Meteorol, 152(2):157–187, August 2014. URL: https://doi.org/10.1007/s10546-014-9919-1 (visited on 2020-07-22), doi:10.1007/s10546-014-9919-1.
Fred C. Bosveld, Peter Baas, Erik van Meijgaard, Evert I. F. de Bruijn, Gert-Jan Steeneveld, and Albert A. M. Holtslag. The Third GABLS Intercomparison Case for Evaluation Studies of Boundary-Layer Models. Part A: Case Selection and Set-Up. Boundary-Layer Meteorol, 152(2):133–156, August 2014. URL: https://doi.org/10.1007/s10546-014-9917-3 (visited on 2020-10-06), doi:10.1007/s10546-014-9917-3.
J. Cuxart, A. A. M. Holtslag, R. J. Beare, E. Bazile, A. Beljaars, A. Cheng, L. Conangla, M. Ek, F. Freedman, R. Hamdi, A. Kerstein, H. Kitagawa, G. Lenderink, D. Lewellen, J. Mailhot, T. Mauritsen, V. Perov, G. Schayes, G-J. Steeneveld, G. Svensson, P. Taylor, W. Weng, S. Wunsch, and K-M. Xu. Single-Column Model Intercomparison for a Stably Stratified Atmospheric Boundary Layer. Boundary-Layer Meteorol, 118(2):273–303, February 2006. URL: https://doi.org/10.1007/s10546-005-3780-1 (visited on 2020-10-06), doi:10.1007/s10546-005-3780-1.
Pawel Gancarski and Javier Sanz Rodrigo. Windbench/gabls3: results of the gabls3 diurnal cycle benchmark for wind energy applications. May 2018. URL: https://zenodo.org/record/1240462 (visited on 2020-10-15), doi:10.5281/zenodo.1240462.
A. a. M. Holtslag, G. Svensson, P. Baas, S. Basu, B. Beare, A. C. M. Beljaars, F. C. Bosveld, J. Cuxart, J. Lindvall, G. J. Steeneveld, M. Tjernström, and B. J. H. Van De Wiel. Stable Atmospheric Boundary Layers and Diurnal Cycles: Challenges for Weather and Climate Models. Bull. Amer. Meteor. Soc., 94(11):1691–1706, November 2013. Publisher: American Meteorological Society. URL: https://journals.ametsoc.org/bams/article/94/11/1691/60301/Stable-Atmospheric-Boundary-Layers-and-Diurnal (visited on 2020-10-14), doi:10.1175/BAMS-D-11-00187.1.
Michal A. Kleczek, Gert-Jan Steeneveld, and Albert A. M. Holtslag. Evaluation of the Weather Research and Forecasting Mesoscale Model for GABLS3: Impact of Boundary-Layer Schemes, Boundary Conditions and Spin-Up. Boundary-Layer Meteorol, 152(2):213–243, August 2014. URL: https://doi.org/10.1007/s10546-014-9925-3 (visited on 2020-10-15), doi:10.1007/s10546-014-9925-3.
J. Sanz Rodrigo, D. Allaerts, M. Avila, J. Barcons, D. Cavar, RA Chávez Arroyo, M. Churchfield, B. Kosovic, J. K. Lundquist, J. Meyers, D. Muñoz Esparza, JMLM Palma, J. M. Tomaszewski, N. Troldborg, MP van der Laan, and C. Veiga Rodrigues. Results of the GABLS3 diurnal-cycle benchmark for wind energy applications. J. Phys.: Conf. Ser., 854:012037, May 2017. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F854%2F1%2F012037 (visited on 2020-07-22), doi:10.1088/1742-6596/854/1/012037.
J. Sanz Rodrigo, M. Churchfield, and B. Kosović. A wind energy benchmark for ABL modelling of a diurnal cycle with a nocturnal low-level jet: GABLS3 revisited. J. Phys.: Conf. Ser., 753:032024, September 2016. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F753%2F3%2F032024 (visited on 2020-10-15), doi:10.1088/1742-6596/753/3/032024.
Javier Sanz Rodrigo. Assessment of meso-micro offline coupling methodology based on driving CFDWind single-column-model with WRF tendencies: the GABLS3 diurnal cycle case. July 2017. URL: https://zenodo.org/record/834356 (visited on 2020-10-15), doi:10.5281/zenodo.834356.
Javier Sanz Rodrigo, Dries Allaerts, Matias Avila, Jordi Barcons, Dalivor Cavar, Roberto Aurelio Chávez Arroyo, Matthiew Churchfield, Branko Kosovic, Julie K Lundquist, Johan Meyers, Domingo Muñoz Esparza, José M L M Palma, Jessica M Tomaszewski, Niels Troldborg, Paul van der Laan, and Carlos Veiga Rodrigues. Results of the GABLS3 diurnal-cycle benchmark for wind energy applications. June 2017. URL: https://zenodo.org/record/4090658 (visited on 2020-10-15), doi:10.5281/zenodo.4090658.
Javier Sanz Rodrigo, Matthew Churchfield, and Branko Kosovic. A wind energy benchmark for ABL modelling of a diurnal cycle with a nocturnal low-level jet: GABLS3 revisited. October 2016. URL: https://zenodo.org/record/4090868 (visited on 2020-10-15), doi:10.5281/zenodo.4090868.
Javier Sanz Rodrigo, Dries Allaerts, Matias Avila, Jordi Barcons, Dalivor Cavar, Roberto A Chávez Arroyo, Mattiew Churchfield, Branko Kosovic, Julie K Lunquist, Johan Meyers, D Muñoz Esparza, Jose MLM Palma, Jessica M Tomaszewski, Niels Troldborg, M Paul Van Der Laan, and Carlos Veiga Rodrigues. GABLS 3 Diurnal-Cycle Benchmark for Wind Energy Applications. Each NetCDF data file corresponds to one simulation submitted to the benchmark. Naming corresponds to the model name as described in the paper., 2017. Publisher: https://b2share.eudat.eu. URL: https://b2share.eudat.eu/records/f5d5a492d8aa4b7998b70abd68f5eae4 (visited on 2020-10-15), doi:10.23728/B2SHARE.F5D5A492D8AA4B7998B70ABD68F5EAE4.
Javier Sanz Rodrigo, Matthew Churchfield, and Branko Kosovic. A methodology for the design and testing of atmospheric boundary layer models for wind energy applications. Wind Energy Science, 2(1):35–54, February 2017. Publisher: Copernicus GmbH. URL: https://wes.copernicus.org/articles/2/35/2017/ (visited on 2020-10-14), doi:https://doi.org/10.5194/wes-2-35-2017.
Andrey Sogachev, Mark Kelly, and Monique Y. Leclerc. Consistent Two-Equation Closure Modelling for Atmospheric Research: Buoyancy and Vegetation Implementations. Boundary-Layer Meteorol, 145(2):307–327, November 2012. URL: https://doi.org/10.1007/s10546-012-9726-5 (visited on 2020-10-15), doi:10.1007/s10546-012-9726-5.
G. Svensson, A. A. M. Holtslag, V. Kumar, T. Mauritsen, G. J. Steeneveld, W. M. Angevine, E. Bazile, A. Beljaars, E. I. F. de Bruijn, A. Cheng, L. Conangla, J. Cuxart, M. Ek, M. J. Falk, F. Freedman, H. Kitagawa, V. E. Larson, A. Lock, J. Mailhot, V. Masson, S. Park, J. Pleim, S. Söderberg, W. Weng, and M. Zampieri. Evaluation of the Diurnal Cycle in the Atmospheric Boundary Layer Over Land as Represented by a Variety of Single-Column Models: The Second GABLS Experiment. Boundary-Layer Meteorol, 140(2):177–206, August 2011. URL: https://doi.org/10.1007/s10546-011-9611-7 (visited on 2020-10-15), doi:10.1007/s10546-011-9611-7.
The marine atmospheric boundary layer is formed through exchanges of momentum moisture and heat across the air-sea interface. Fig. 21 shows a schematic of the physical processes that take place in the coupled atmospheric-ocean boundary layer [ECC+07]. Mesoscale variability and wave-induced effects produce deviations from land-based turbulence spectra and flux-profile relationships used in ABL parameterizations [SHB+99]. The coastal region, where most offshore wind energy is deployed, is subject to large mesoscale variability influenced by coastal topography and temperature gradients between land and sea [DOMS15]. These heterogeneous conditions generate important horizontal wind speed gradients breaking MOST assumptions on which many wake models are based. Furthermore, frequent stable conditions develop due to large land-see temperature contrast producing large-shear low-level jets (LLJ) in shallow boundary-layers and gravity waves that increase wind farm global blockage [AM18]. In extreme wind conditions, wave-induced shear-stress becomes dominant favouring the use of a wind-wave coupled model [LDB+19].
The NEWA project has produced two benchmarks suitable for the assessment of the coastal-offshore boundary layer. The Ferry-Lidar experiment consist on following a ferry-mounted profiling lidar, for a period of 4 months, along a regular route in the Southern Baltic Sea between Kiel (Germany) and Klaipeda (Lithuania). The RUNE experiment comprises measurements from 8 lidars and a buoy to measure the evolution of the wind speed along a ~8 km fetch.
The Ferry-Lidar Marine Boundary-Layer (Ferry-Lidar)¶
Status¶Scope and Objectives¶Background¶Measurement Campaign¶Input Data¶Validation Data¶Output Data¶Remarks¶References¶
GCDW18
Julia Gottschall, Eleonora Catalano, Martin Dörenkämper, and Björn Witha. The NEWA Ferry Lidar Experiment: Measuring Mesoscale Winds in the Southern Baltic Sea. Remote Sensing, 10(10):1620, October 2018. Number: 10 Publisher: Multidisciplinary Digital Publishing Institute. URL: https://www.mdpi.com/2072-4292/10/10/1620 (visited on 2020-07-22), doi:10.3390/rs10101620.
MAA+17
J. Mann, N. Angelou, J. Arnqvist, D. Callies, E. Cantero, R. Chávez Arroyo, M. Courtney, J. Cuxart, E. Dellwik, J. Gottschall, S. Ivanell, P. Kühn, G. Lea, J. C. Matos, J. M. L. M. Palma, L. Pauscher, A. Peña, J. Sanz Rodrigo, S. Söderberg, N. Vasiljevic, and C. Veiga Rodrigues. Complex terrain experiments in the New European Wind Atlas. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 375(2091):20160101, April 2017. Publisher: Royal Society. URL: https://royalsocietypublishing.org/doi/full/10.1098/rsta.2016.0101 (visited on 2020-07-22), doi:10.1098/rsta.2016.0101.
A canopy is considered when the roughness elements over the ground surface are of significant size compared to the height of interest (e.g. hub-height). Here we shall focus on forest canopies although the term could be also applied for a urban canopy or a wind farm canopy. The classical interpretation of the flow over a forest canopy is that of a displaced logarithmic profile originating by the momentum absortion through aerodynamic drag across the depth of the canopy [JCJJ94], where the displacement height depends on the distribution of the drag through the foliage. Within the canopy the flow is highly turbulent and heterogeneous which prevents us from extending MOST to the canopy layer as if it was a rough lower boundary. A more realistic approach requires to model explicitely the distribution of drag and the energy balance between the surface and the air aloft to characterize the flow above and within the canopy.
The structure of the turbulent flow in horizontally homogeneous canopies have been the object of study of numerous experiments ranging from wind tunnel models (e.g. [BFR94]) to tall forests [JCJJ94]. All the profiles display a characteristic inflexion point near the canopy top which separates the canopy flow from the boundary layer profile above. A constant shear stress region is present in the free-stream which decreases rapidly as momentum is absorbed by the canopy.
Heterogeneous forest canopies combine patches of trees of different heights and foliage density that typically change throughout the year. Aerial lidar scans are used to map the tree height and plant area density [BBT+15]. Highly heterogeneous conditions happen at forest edges as the flow transitions through an internal boundary layer towards a developed canopy boundary layer [DBM14][BDBD17].
The Ryningsnäs experiment was used in the NEWA project to validate ABL models in neutral conditions [IAA+18] considering mean vertical profiles of wind speed and turbulence intensity for different wind direction sectors. A follow-up experiment, Hornamossen, studied the mean flow along a transect of 9 remote sensing profilers and a reference 180-m met mast over undulated forested terrain [MAA+17]. The Østerild Balconies experiment deployed two horizontally scanning lidars on vertical masts at 50 and 200 m to characterize the heteorogeneous mean flow above the canopy over a relatively flat and semi-forested terrain [KMDV18].
Status¶Scope and Objectives¶Background¶Measurement Campaign¶Input Data¶Validation Data¶Output Data¶Remarks¶References¶
AOEI19
Johan Arnqvist, H. Olivares-Espinosa, and S. Ivanell. Investigation of Turbulence Accuracy When Modeling Wind in Realistic Forests Using LES. In Ramis Örlü, Alessandro Talamelli, Joachim Peinke, and Martin Oberlack, editors, Progress in Turbulence VIII, Springer Proceedings in Physics, 291–296. Cham, 2019. Springer International Publishing. doi:10.1007/978-3-030-22196-6_46.
The Hornamossen Diurnal-Cycle in Forested Undulating Terrain (Hornamossen)¶
Status¶Scope and Objectives¶Background¶Measurement Campaign¶Input Data¶Validation Data¶Output Data¶Remarks¶References¶
MAA+17
J. Mann, N. Angelou, J. Arnqvist, D. Callies, E. Cantero, R. Chávez Arroyo, M. Courtney, J. Cuxart, E. Dellwik, J. Gottschall, S. Ivanell, P. Kühn, G. Lea, J. C. Matos, J. M. L. M. Palma, L. Pauscher, A. Peña, J. Sanz Rodrigo, S. Söderberg, N. Vasiljevic, and C. Veiga Rodrigues. Complex terrain experiments in the New European Wind Atlas. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 375(2091):20160101, April 2017. Publisher: Royal Society. URL: https://royalsocietypublishing.org/doi/full/10.1098/rsta.2016.0101 (visited on 2020-07-22), doi:10.1098/rsta.2016.0101.
To study the influence of terrain elevation changes on the ABL we first study the flow over isolated hills, i.e. when the inflow conditions are relatively homogeneous and the height of the hill is small compared to the ABL height (~100 m, where surface-layer turbulence dominates the flow). These conditions have been traditionally used to validate linearized flow models since pioneering work from Jackson and Hunt (1975) [JH75] that led to the Askervein experiment in 1982 and 1983 [TT87]. This is a 116-m high hill in Scothland with gentle slopes (i.e. less than ~30% to prevent flow separation) and homogeneous inflow from the SW, where a quasi-steady flow case in neutral conditions was generated. This case became a golden benchmark to test flow-over-hill models until the Bolund hill experiment in 2007 [BBC+09][BMB+11]: a 12-m high isolated ridge with steep terrain in the prevailing wind direction, suitable for testing non-linear flow models still in surface-layer and predominanantly neutral conditions [BSB+11].
The idealized inflow conditions assumed in flow-over-hill studies are suitable for wind tunnel experiments. These are adequate for parametric testing of the flow under different hill shapes, surface roughnesses and stability coditions, e.g. [FRBA90], [KP00], [RAV+04], [RV05], [WPA11].
The NEWA experiment at the Röderser Berg hill near Kassel is the most recent field campaign in the isolated hills category in the wind energy context. This ~200 m hill, covered by a forest of 20-30 m tree-heights, includes two 200-m tall masts at the inflow and hilltop and doppler-lidar longitudinal profiles across the hill to measure the speed-up at two heights (80 and 135 m) [DWG19].
Fig. 22 shows the results of two RANS simulations with similar performance. While the mean flow is well reproduced, the tke is underestimated at the lee side of the hill. This deficiency is attributed to the limited capability of isotropic turbulence models in wake flows as much better agreement is found with LES models [SLPC07].
The benchmark is intended for microscale models in neutral conditions. We shall focus first on the classical 210º flow case which presents a smooth transition to the hill top from relatively homogeneous inflow conditions.
The objectives are:
Test microscale model settings for complex terrain.
Evaluate turbulence models in a test site with reasonably well defined boundary conditions.
Quantify performance metrics for future reference.
Further analysis can follow on less ideal flow cases, namely:
The Askervein hill project can be considered the cornerstone of boundary layer flow over hills. It is based on two filed campaigns conducted in 1982 and 1983 on and around the Askervein hill, a 116 m high (126 m above sea level) hill on the west coast of the island of South Uist in the Outer Hebrides, Scotland (Fig. 23). The hill is isolated in all wind directions but the NE-E sector. To the SW there is a flat uniform fetch of 3-4 km to the coastline where there are sand dunes and low cliffs. A uniform roughness of 0.03 m is assumed all over the hill. The smooth slopes of the hill, generally less than 20% with some small areas reaching 30%, ensures fully attached flow most of the time, being a rather friendly site for flow models.
Over 50 towers were deployed and instrumented for wind speed and turbulence measurements. 35 of them consisted on 10 m masts, instrumented with cup anemometers only at 10m, configuring two arrays across the mayor axis of the hill (lines A and AA), in the prevailing wind direction from SW, and one array along the minor axis of the hill (line B). TALA kites (TK runs) were used during some periods to provide upstream profiles up to ~500 m. Regular AIRsonde and upper air soundings were used to define the state of the atmosphere. In the 1983 experiment, two 50 m towers (at a reference position RS, 3 km upstream in the SSW direction, and at the hill top HT, both with cup and Gill UVW anemometers), a 30 m tower (at the base of the hill BRE), a 16 m tower at CP’ (near CP with Gill and cup anemometers), and thirteen 10 m towers were instrumented for 3-component turbulence measurements. Exact tower positions are given in the ASK83 document. In addition to the anemometry, other instruments were deployed to provide background temperature, humidity, precipitation and pressure data. In particular, in the ASK83 campaign, the temperature difference between 4.9 and 16.9 m at RS was monitored in order to estimate the Richardson number.
Two field campaigns were conducted during September-October 1982 (ASK82 campaign, [TT83]) and 1983 (ASK83 campaign, [TT85]). The velocities from the 10 m masts were averaged over 30 min runs for selected periods, usually of 2 hr total duration, to obtain mean flow profiles (MF runs). Turbulence data were recorded for selected periods only, also processed as 30 min blocks and combined to form 2 hr runs (TU runs). The ASK82 campaign comprises 24 hours of moderate-to-strong surface winds from the undisturbed wind direction sectors leading to 11 MF runs. The ASK83 campaign comprises a 16-day period with a total of 44 MF and 19 TU runs, almost all in near-neutral atmosphere. Further information about the campaigns and an inventory of all the measurements are summarized by Taylor and Teunissen (1987) [TT87].
Salmon et al (1988) [SBH+88] presented results on the variations in mean wind speed at fixed points above the ground. An analysis of the vertical profiles of mean wind and integral turbulence statistics at the reference masts is reported by Mickle et al (1988) [MCH+88].
Numerous papers have been published on atmospheric models of the Askervein hill test case, almost all of them dealing with the 210º wind direction case, almost aligned with lines A and AA. Walmsley and Taylor (1996) [WT96] presented both numerical and wind tunnel model results in a survey of the intensive research developed in the first decade after the field campaign. Linear models as the spectral model of Beljaars et al. (1987) [BWT87], perform well in predicting the mean flow observations in the upwind slopes and at the hilltop but fail in the lee side of the hill. These difficulties were significantly overcome by introducing non-linear terms in the spectral model [XT92].
The wind tunnel results of the early studies showed an important dependency of the flow field on the model roughness: while the best agreement on the windward side and the summit of the hill were obtained with smooth models, the performance in the lee side was better with rough models. The rough physical model also provided better fit to turbulence variables.
The application of CFD models in the simulation of the Askervein hill case has been the main activity of the last two decades since the pioneering works of Raithby et al. (1987) [RST87], who simulated an isolated Askervein hill with a mesh of 20x20x19 = 7600 cells. Classical CFD-related issues have been addressed including: effect of topographic detail and domain dimensions, grid resolution, turbulence closure, and inlet and terrain boundary conditions. Kim and Patel (2000) [KP00] tested different steady RANS turbulence models and found the best performance with the RNG version of \(k-\epsilon\). Castro et al. (2003) [CPSL03] present the results of steady and unsteady RANS \(k-\epsilon\) turbulence model at different grid resolutions, showing good performance in the mean flow even with coarse grids. The influence of downstream hills, for the 210º case, was also assessed concluding that their influence on the flow at the lee side of the Askervein hill was not important. Variable roughness and transient simulations presented the best results in the predictions of the unsteady flow field of the lee side of the hill.
Undheim et al. (2006) [UAB06] used a commercial CFD solver based on steady \(k-\epsilon\) closure with Coriolis effects included. The inlet boundary conditions were defined by simulating a homogeneous 1D atmospheric boundary layer. Good performance is observed in predicting the mean flow field but, as found by previous RANS-based studies, the turbulence in the lee side of the hill is underestimated. Grid dependency simulations were conducted varying both horizontal and vertical resolution. Vertical resolution showed larger influence, particularly regarding the relation between first-cell height and wall the functions. Vertical resolution is pointed out as the key issue related to the simulation of turbulence in the wake of the hill.
Silva Lopes et al. (2007) [SLPC07] performed LES simulations of the Askervein 210º run obtaining good solution for the mean flow and better results on turbulence profiles than with RANS \(k-\epsilon\)[CPSL03]. However grid convergence was not achieved in the lee side of the hill. Bechmann (2006) [Bec07] also performed LES simulations of this case, using RANS in the near wall region. Compared to a full RANS simulation, the LES results showed improvement in predicting the hilltop speed-up and the turbulent kinetic energy in the lee side of the hill, where RANS showed large under-predictions.
Digitized map covering an area of 15x19 km based on 1:25000 maps (elevation lines every 10 m). Higher resolution digitized map of 2.5x2.5 km of the Askervein hill at 1:5000 (lines every 2 m).
Roughness map of the 15x19 km based on 1:25000 maps. Roughness levels: 0.0002 m (water bodies), 0.4 m (build-up area) and 0.03 m (background roughness).
Coordinates of met masts along lines A, AA and B and at RS, CP and HT.
Inlet conditions shall be based on MOST profiles fitted to RS data.
The validation dataset is based on ensemble mean values of:
Fractional-Speedup-Ratio (FSR) and normalized added turbulent kinetic energy (TKE) with respect to the reference inlet position, at 10 m above ground level along mast lines A, AA and B.
FSR and TKE vertical profiles at the reference (RS), hilltop (HT) and centre point (CP) positions
Velocity and TKE values will be normalized with respect to the RS position. The validation dataset includes measurements during the following runs [TT85]):
where MF runs corresponds to mean flow measurements, TU runs corresponds to turbulence runs and TK to TALA kite runs. Hence, the 90º case does not have turbulence data and shall be used to assess the sensitivity of the mean flow to the wind direction variability in hill-induced wake conditions.
The following simulation runs are considered, corresponding to the different wind directions of the measurements:
Run 1: 210º, fine-tuning
Run 2: 130º, blind
Run 3: 90º, blind
Run 4: 95º, blind
Run 5: 100º, blind
Run 6: 105º, blind
all in neutral conditions. The computational grid should include the hills behind Askervein. A grid dependency study should be conducted in order to assess the model sensitivity to the selected grid design. This study should be described in a self-assessment report and only the outputs from final runs should be provided.
The origin of the coordinate system should be placed at the HT position with X aligned with the incoming wind direction, Z pointing up and Y perpendicular to the XZ plane in a right-handed system.
The simulated validation profiles consist on horizontal profiles along lines A, AA and B at 10 m height above ground level and vertical profiles at RS, HT and CP position, of velocity components (U,V,W), turbulence kinetic energy (tke) and dissipation rate (tdr). The profiles should traverse the simulated domain from boundary to boundary. Hence, the required outputs are, in this order: X(m), Y(m), Z(m), U(m/s), V(m/s), W(m/s), tke(m2/s2), tdr(m2/s3).
Use the file naming and format convention described in the Windbench user’s guide with profID = prof#, where # = [A,AA,B,RS,HT,CP], i.e. 6 output files per user and model run.
Run 1, with validation data provided together with the inputs. This simulation shall be used to fine-tune the model in order to match the validation dataset as close as possible. In order to evaluate the added value of model fine-tuning it is important that you describe how this is performed. Please report on the deviations with respect to default settings if validation data were not available a priori (blind conditions).
Runs 2 to 6: Based on the model parameterization of the first run, provide simulations for the other wind directions in blind conditions. The validation data will be provided as soon as the simulation results are submitted.
There are no guidelines on the definition of the computational mesh so please describe how you integrate grid dependency in the evaluation process. Bear in mind that grid sensitivity will be direction dependent.
A. Bechmann and N. N. Sørensen. Hybrid RANS/LES method for wind flow over complex terrain. Wind Energy, 13(1):36–50, 2010. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/we.346. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/we.346 (visited on 2020-07-23), doi:10.1002/we.346.
Andreas Bechmann. Large-eddy simulation of atmospheric flow over complex terrain. PhD thesis, Technical University of Denmark, August 2007. Risø-PhD-28(EN).
A. C. M. Beljaars, J. L. Walmsley, and P. A. Taylor. A mixed spectral finite-difference model for neutrally stratified boundary-layer flow over roughness changes and topography. Boundary-Layer Meteorol, 38(3):273–303, February 1987. URL: https://doi.org/10.1007/BF00122448 (visited on 2020-07-23), doi:10.1007/BF00122448.
F. A. Castro, J. M. L. M. Palma, and A. Silva Lopes. Simulation of the Askervein Flow. Part 1: Reynolds Averaged Navier–Stokes Equations (k∈ Turbulence Model). Boundary-Layer Meteorology, 107(3):501–530, June 2003. URL: https://doi.org/10.1023/A:1022818327584 (visited on 2020-07-23), doi:10.1023/A:1022818327584.
H. G. Kim and V. C. Patel. Test Of Turbulence Models For Wind Flow Over Terrain With Separation And Recirculation. Boundary-Layer Meteorology, 94(1):5–21, January 2000. URL: https://doi.org/10.1023/A:1002450414410 (visited on 2020-07-23), doi:10.1023/A:1002450414410.
R. E. Mickle, N. J. Cook, A. M. Hoff, N. O. Jensen, J. R. Salmon, P. A. Taylor, G. Tetzlaff, and H. W. Teunissen. The Askervein Hill Project: Vertical profiles of wind and turbulence. Boundary-Layer Meteorol, 43(1):143–169, April 1988. URL: https://doi.org/10.1007/BF00153977 (visited on 2020-07-23), doi:10.1007/BF00153977.
G. D. Raithby, G. D. Stubley, and P. A. Taylor. The Askervein hill project: A finite control volume prediction of three-dimensional flows over the hill. Boundary-Layer Meteorol, 39(3):247–267, May 1987. URL: https://doi.org/10.1007/BF00116121 (visited on 2020-07-23), doi:10.1007/BF00116121.
Javier Sanz Rodrigo, Pawel Gancarski, Roberto Chavez Arroyo, Patrick Moriarty, Matthew Chuchfield, Jonathan W. Naughton, Kurt S. Hansen, Ewan Machefaux, Tilman Koblitz, Eoghan Maguire, Francesco Castellani, Ludovico Terzi, Simon-Philippe Breton, Yuko Ueda, John Prospathopoulos, Gregory S. Oxley, Carlos Peralta, Xiadong Zhang, and Björn Witha. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. J. Phys.: Conf. Ser., 524:012105, June 2014. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F524%2F1%2F012105 (visited on 2020-10-14), doi:10.1088/1742-6596/524/1/012105.
Javier Sanz Rodrigo and Patrick Moriarty. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. June 2014. URL: https://zenodo.org/record/4088287 (visited on 2020-10-14), doi:10.5281/zenodo.4088287.
J. R. Salmon, A. J. Bowen, A. M. Hoff, R. Johnson, R. E. Mickle, P. A. Taylor, G. Tetzlaff, and J. L. Walmsley. The Askervein Hill Project: Mean wind variations at fixed heights above ground. Boundary-Layer Meteorol, 43(3):247–271, May 1988. URL: https://doi.org/10.1007/BF00128406 (visited on 2020-07-23), doi:10.1007/BF00128406.
P. Taylor and H. W. Teunissen. Askervein ’82: report on the September/October 1982 experiment to study boundary layer flow over Askervein, South Uist. Technical Report MSRS-83-8, Meteorological Services Research Branch, Atmospheric Environment Service, Downsview, Ontario, Canada, 1983. URL: http://www.yorku.ca/pat/research/Askervein/ASK82.pdf (visited on 2020-10-16).
P. Taylor and H. W. Teunissen. The Askervein Hill Project: report on the September/October 1983, main field experiment. Technical Report MSRS-84-6, Meteorological Services Research Branch, Atmospheric Environment Service, Downsview, Ontario, Canada, 1985. URL: http://www.yorku.ca/pat/research/Askervein/ASK83.pdf (visited on 2020-10-16).
P. A. Taylor and H. W. Teunissen. The Askervein Hill project: Overview and background data. Boundary-Layer Meteorol, 39(1):15–39, April 1987. URL: https://doi.org/10.1007/BF00121863 (visited on 2020-07-23), doi:10.1007/BF00121863.
John L. Walmsley and Peter A. Taylor. Boundary-layer flow over topography: Impacts of the Askervein study. Boundary-Layer Meteorol, 78(3):291–320, March 1996. URL: https://doi.org/10.1007/BF00120939 (visited on 2020-07-23), doi:10.1007/BF00120939.
Dapeng Xu and Peter A. Taylor. A non-linear extension of the mixed spectral finite difference model for neutrally stratified turbulent flow over topography. Boundary-Layer Meteorol, 59(1):177–186, April 1992. URL: https://doi.org/10.1007/BF00120693 (visited on 2020-07-23), doi:10.1007/BF00120693.
The results are consistent with the original blind test with much better results at 5 m than at 2 m. Fig. 22 shows mean profiles along the B5 and A5 profiles where the \(k-\epsilon\) model shows superior performance than the \(k-\omega\) model due to a much finer grid around the escarpment zone.
The Bolund blind comparison was organized by DTU in 2009. You can find input and validation data as well as results of the blind test in project website[BSB+11].
Highlights
In total 49 different simulations were submitted, composed of 3 physical models, 9 linearized numerical models and 37 CFD models (5 LES, 7 RANS 1-equation and 25 RANS 2-equation). The physical models predicted reasonably well the mean velocity profiles but under-predicted the turbulent kinetic energy. Linear models produced the worse results as they were not capable of reproducing the flow around the steep escarpment. RANS models provided the best results although the spread of the simulations was quite big, indicating user dependencies especially regarding mesh generation. LES-based models had problems but presented promising results with regard to turbulence modelling in the flow separation area just after the escarpment. The average error in the simulation of the mean wind speeds over all the measurement locations was 13-17% in the top 10 best models. This error dropped to 4-10% if only the measurements at 5 m were considered.
The benchmark revisits the blind test of 2009 now allowing the participants to optimize their models to obtain the best match to the validation dataset.
The objectives are:
Test model fine-tuning strategies that will be applied in complex terrain sites.
Evaluate turbulence models in a test site with well defined boundary conditions.
Bolund is a 12 m high, 130 m long and 75 m wide isolated hill situated to the North of RisøDTU in Roskilde Fjord, Denmark. It is surrounded by water in all directions except to the E, where a narrow isthmus leads to the mainland. The hill is characterized by a uniform roughness of 0.015 m and surrounded by water with a roughness length of 0.0003 m. An almost vertical escarpment in the prevailing W-SW sector ensures flow separation in the windward edge resulting in a complex flow field, quite challenging for flow models.
The masts are positioned along two lines: A and B. Two additional masts (M0 and M9) were installed to measure the incoming undisturbed flow for westerly and easterly winds respectively. Mast M9 is placed in the coastline, where the roughness length is again 0.015 m. The masts are equipped with 23 sonic (Metek USA 1-Basic) and 12 cup anemometers (Risø Wind Sensor P2546) at heights between 2 and 15 m.
The Bolund experiment comprises a measurement campaign of three months between 2007 and 2008 [BBC+09][BMB+11]. During the measurement campaign the absolute water level was monitored, which covered the isthmus most of the time. The campaign was designed for W-SW winds where the fetch ranges 4 to 7 km, ensuring undisturbed velocity profiles over water. The E sector is more difficult to characterize due to a more the heterogeneous land cover. The prevailing stability regime was neutral to slightly stable conditions (1/L < 0.04). The data was averaged over 30 min periods.
The original blind test after the experiment was conducted by DTU in 2009 [BSB+11]. Recent work on the Bolund hill include RANS simulations [PPC12], LES simulations [DHF+13] and wind tunnel experiments [YCTPA15][CCA+16].
The validation dataset is composed of mean flow and turbulence data from cup and sonic anemometers at 10 met masts. Ensemble averages of 10 min averaged samples within ±8º wind direction sector, with wind speeds between 5 and 12 m/s at 5 m level and under neutral conditions (\(|1/L| < 0.004 m^{-1}\)) at the upstream masts, were used to derive the validation datasets which consists on:
Fractional-Speedup-Ratio (FSR) and normalized added turbulent kinetic energy (TKE) with respect to the reference inlet position, at 2 and 5 m above ground level along mast lines A and B
FSR and TKE vertical profiles at mast positions.
Velocity and TKE values will be normalized with the upstream friction velocity at the reference mast as in Bechmann et al. (2011). The validation dataset includes mean and standard deviation statistics from the ensemble profiles.
or by best fit to the measured inlet profiles (at M0 for runs 1,2 and 3 and M9 for run 4) if the participant considers that this can improve the results. The computational domain must extend at least to X = ±400 m in order to include the coastline to the East and make sure that the hill wake is completely covered. The origin of the coordinate system should be placed at M3 position with X pointing East, Y pointing North and Z pointing up.
The simulated validation profiles consist on horizontal profiles along lines A and B at 2 and 5 m height and vertical profiles at mast positions of velocity components (U,V,W), turbulence kinetic energy (tke), dissipation rate (tdr), friction velocity (ust) and kinematic momentum fluxes (uu, vv, ww). The profiles should traverse the simulated domain from boundary to boundary. Hence, the required outputs are, in this order: X(m), Y(m), Z(m), U(m/s), V(m/s), W(m/s), tke(m2/s2), tdr(m2/s3), us(m/s), uu(m2/s2), vv(m2/s2), ww(m2/s2).
Use the file naming and format convention described in the Windbench user’s guide with profID = prof#, where # = [M0,M1,M2,M3,M5,M6,M7,M8,M9,A2,A5,B2,B5], i.e. 13 output files per user and model run. Additionally, for those users that participated in the blind test of 2009, please provide the output files that were obtained at that time. This will allow an assessment of the added value of onsite measurements for model tuning. Please follow the same format described before but with a BenchmarkID = Bolund_blind2009 to differentiate between the two sets of simulations.
In order to evaluate the added value of model fine-tuning it is important that you describe how this is performed. Please report on the deviations with respect to default settings (those of the blind test). There are no guidelines on the definition of the computational mesh since this can have an important influence in the fine-tuning aspects of the model. Please describe how you integrate grid dependency in the evaluation process.
A. Bechmann, N. N. Sørensen, J. Berg, J. Mann, and P.-E. Réthoré. The Bolund Experiment, Part II: Blind Comparison of Microscale Flow Models. Boundary-Layer Meteorol, 141(2):245, August 2011. URL: https://doi.org/10.1007/s10546-011-9637-x (visited on 2020-07-23), doi:10.1007/s10546-011-9637-x.
J. Berg, J. Mann, A. Bechmann, M. S. Courtney, and H. E. Jørgensen. The Bolund Experiment, Part I: Flow Over a Steep, Three-Dimensional Hill. Boundary-Layer Meteorol, 141(2):219, July 2011. URL: https://doi.org/10.1007/s10546-011-9636-y (visited on 2020-07-23), doi:10.1007/s10546-011-9636-y.
Boris Conan, Ashvinkumar Chaudhari, Sandrine Aubrun, Jeroen van Beeck, Jari Hämäläinen, and Antti Hellsten. Experimental and Numerical Modelling of Flow over Complex Terrain: The Bolund Hill. Boundary-Layer Meteorol, 158(2):183–208, February 2016. URL: https://doi.org/10.1007/s10546-015-0082-0 (visited on 2020-07-23), doi:10.1007/s10546-015-0082-0.
Marc Diebold, Chad Higgins, Jiannong Fang, Andreas Bechmann, and Marc B. Parlange. Flow over Hills: A Large-Eddy Simulation of the Bolund Case. Boundary-Layer Meteorol, 148(1):177–194, July 2013. URL: https://doi.org/10.1007/s10546-013-9807-0 (visited on 2020-07-23), doi:10.1007/s10546-013-9807-0.
Javier Sanz Rodrigo, Pawel Gancarski, Roberto Chavez Arroyo, Patrick Moriarty, Matthew Chuchfield, Jonathan W. Naughton, Kurt S. Hansen, Ewan Machefaux, Tilman Koblitz, Eoghan Maguire, Francesco Castellani, Ludovico Terzi, Simon-Philippe Breton, Yuko Ueda, John Prospathopoulos, Gregory S. Oxley, Carlos Peralta, Xiadong Zhang, and Björn Witha. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. J. Phys.: Conf. Ser., 524:012105, June 2014. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F524%2F1%2F012105 (visited on 2020-10-14), doi:10.1088/1742-6596/524/1/012105.
Javier Sanz Rodrigo and Patrick Moriarty. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. June 2014. URL: https://zenodo.org/record/4088287 (visited on 2020-10-14), doi:10.5281/zenodo.4088287.
T.S. Yeow, A. Cuerva-Tejero, and J. Perez-Alvarez. Reproducing the Bolund experiment in wind tunnel. Wind Energy, 18(1):153–169, 2015. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/we.1688. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/we.1688 (visited on 2020-07-23), doi:10.1002/we.1688.
The Rödeser Berg Forested Smooth Hill (Rödeser Berg)¶
Status¶Scope and Objectives¶Background¶Measurement Campaign¶Input Data¶Validation Data¶Output Data¶Remarks¶References¶
According to the IEC 61400-12-1 standard [ITC8817], in the context of power performance testing, complex terrain is terrain surrounding the test site that features significant variations in topography and terrain obstacles that may cause flow distortion, i.e. significant variations in the wind conditions that affect turbine performance (wind shear, turbulence intensity, etc). To categorize terrain complexity, three types of complex terrain are defined [ITC8817]:
Type A: does not have significant changes in elevation relative to the hub height or particularly steep slopes over long distances. Examples of Type A terrain include gentle rolling hills, and turbine located on a ridge facing a plane.
Type B: includes mountains, ridgelines, large hills and hilly sites with moderate to steepy sloping terrain and significant changes in elevation relative to the hub height.
Type C: typically have a steep terrain feature such as a mountain or canyon that may cause flow separation directly upwind of the site of interest. These conditions create drastic changes in wind conditions
These definitions are relative to the terrain conditions seen at a target wind turbine site and, therefore, depend on the wind direction sector under consideration.
Terrain complexity has been quantified using the ruggedness index RIX which measures the fraction of terrain surface that is steeper than a critical slope of 30-40%. This index is related to the likelihood of flow separation, an important factor in the performance of flow models [MTL08]. In effect, the difference between the RIX at the reference (measured) site and that at the predicted site, \(\Delta RIX\) shows positive correlation with the prediction error of a linearized model when the flow is dominated by terrain effects, i.e. without much influence from thermal effects, drastic roughness changes, forest canopy effects or mesoscale effects [MTL08].
Fig. 21 shows a schematic of the Perdigão double-hill experiment illustrating the most relevant physical phenomena [FMP+19]. While the flow near the surface is dominated by terrain effects, wind conditions at turbine-relevant heights are modulated by mesoscale thermal and (synoptic) topographic forcings. Thermal circulation effects due to differences in temperature between the hilltop and the valley cause upslope (anabatic) and katabatic (downslope) winds driven by buoyancy forces. Under synoptic conditions the flow is determined by atmospheric stability. In stable conditions, turbulence mixing is reduced mitigating flow separation but favouring the generation of multiple flow phenomena: lee waves, hydraulic jumps, shear layers, upstream blocking, etc. In neutral and unstable conditions terrain-induced flow separation is more likely producing recirculation in the lee side and other unsteady phenomena due to the interaction with downstream topography.
Besides Perdigão, the NEWA project produced a follow-up experiment in complex terrain at the Alaiz site [SMV+20] to study the interaction between the Tajonar ridge, of similar height as Perdigão hills (~250 m), and the Alaiz mountain range (~700 m), both separated by a ~ 6-km wide valley. The high altitude of Alaiz results in larger interaction with the ABL structure and mesoscale flow.
This section describes benchmarks in the application space, where flow models are integrated with the statistics of the long-term wind climate to predict the quantities of interest that are relevant for the inteded uses of the model (see Intended Use section). Before running these benchmarks, flow models should have demonstrated their predictive capacity by validating as many flow cases as possible from the Benchmarks on Flow Phenomena section. This way, it will be easier to understand the limitations of the model and relate them to the resulting prediction bias. In other words, the objective is to quantify the impact that the formal V&V process has on the applications of interest and highlight gaps that should be addressed in the next round of targeted experiments and validation campaigns.
The assessment shall be done on as many sites as possible to cover a wide range of operational conditions. This will require the provision of measurement campaigns from industry to complement the publicly available experimental campaigns.
Assessment of Wind Resource, Energy Yield and Site Suitability¶
The NEWA project offers a number of public long-term measurement campaigns that can be used to validate AEP and long-term wind conditions in different European wind climates and topographic conditions. A baseline case in horizontally homogeneous conditions is based at the Cabauw Experimental Site for Atmospheric Research (CESAR) in the Netherlands.
Annual Energy Yield Assessment and Wind Conditions at Cabauw (Cabauw)¶
The Cabauw benchmark was organized within Wakebench Phase 2 with support from the NEWA and MesoWake EU projects. The results were presented at the Torque 2018 conference.
The first phase of the NEWA Meso-Micro Challenge was launched to establish a validation process that enables model developers to design wind resource downscaling methodologies. Initial results for the Cabauw site correspond to RANS and LES microscale ABL models coupled to mesoscale using the tendencies approach, which was previously tested in the GABLS3 benchmark [RAA+17]. Initial results correspond to a one-year (2006) integration of the models to obtain time-series of wind conditions and annual energy production. The results show consistency at reproducing the annual wind climate distribution and mean velocity profiles. Mean turbulence profiles are also well predicted by the LES model SP-Wind which can then be used as a reference to improve RANS models. Fig. 27 shows bin-averaged quantities of interest and bias for this model. The wind resource is underestimated consistently with the input mesoscale data. It also underestimates the large wind shear and overestimate turbulence in stable conditions, probably due to the relatively coarse resolution of the model (15 m).
This first phase of the NEWA Meso-Micro Challenge deals with two sites in horizontally homogeneous conditions:
Cabauw, onshore, and
Fino1 offshore
In these conditions single-column models can be used cost-effectively as proxy for 3D RANS models. This will provide a more efficient approach to test statistical methodologies that can be later applied to heterogeneous sites in 3D.
The objectives of the challenge apply in this first phase as follows:
Test meso-micro methodologies consistently for two sites and map accuracy vs cost for relevant quantities of interest, notably for annual energy production (AEP) and site suitability parameters.
Establish open-access practices for the assessment of these methodologies that can determine the added value of meso-micro with respect to conventional methods based on microscale modeling only.
Compare methodologies for uncertainty estimation of gross AEP and discuss their adequacy to the wind resource assessment process used by industry.
This challenge is organized in the context of the New European Wind Atlas (NEWA) project, whose overarching goal is to produce a seamless high resolution wind atlas for Europe. The wind atlas methodology will be based on a mesoscale to microscale (meso-micro) model-chain, validated with dedicated experiments as well as other observational databases from public and private sources. Wind resource assessment is related to the development of wind farms and implies the prediction of long-term wind statistics, notably the annual energy prediction (AEP).
In the development of meso-micro methodologies for wind resource assessment there is a tradeoff to be made between modeling fidelity and its associated cost to yield the required accuracy for the intended use (Fig. 28). Accuracy is a qualitative concept that is used here to define the closeness of agreement between the predicted quantity of interest and the true value in the real word. Considering wind resource assessment applications, accuracy should gradually improve from the early-stage prospecting phase to the project financing phase, i.e. from planning to bankable accuracy. This process will hopefully remove the bias and reduce the uncertainty of the assessment to desired financial limits. This typically implies using off-the-shelf wind atlas products during early planning phase to design tools of increasing fidelity as the project matures. The required fidelity will depend on the complexity of the site as indicated in Figure 1 and is capped by the maximum allocated cost in terms of computing time.
Illustration of the process of improved accuracy ΔU from planning to bankable thresholds against the maximum allocated computing time cost for different site/flow complexities.¶
A hierarchy of meso-micro methodologies for wind resource assessment is illustrated in Figure 2, ranging from the Global Wind Atlas lower-end of modeling fidelity, where the WAsP downscaling method is used directly from global reanalysis data without mesoscale modeling, to the dynamic coupling of a mesoscale model with a Computational Fluid Dynamics (CFD) model based on large-eddy simulation (LES) in the higher-end of modeling fidelity. The Weather Research and Forecasting (WRF) mesoscale model will be used in NEWA to produce the wind atlas and, therefore, it is explicitly mentioned in Fig. 29, also as the most popular choice for a mesoscale model. Between these limits, a hierarchy of methodologies is established depending on the type of coupling and the type of CFD model. Since the application demands statistical quantities of interest, we shall leave dynamical coupling methods out of the design tools range. In the context of NEWA, wind farm design tools will be based on statistical downscaling methodologies that combine WRF outputs with steady or unsteady Reynolds-Averaged Navier Stokes (RANS) microscale CFD model simulations that include thermal stratification.
Hierarchy of meso-micro methodologies for wind resource assessment classified in terms of the type of coupling and typical intended use.¶
Similarly, uncertainty quantification can also have different levels of fidelity depending on how rigorous is the analysis; from ad-hoc engineering methods to formal UQ probabilistic methods.
Input data including tendencies for Cabauw can be found in this repository.
Surface roughness
Microscale models shall consider a uniform roughness length for Cabauw of 0.15 m, as in the GABLS3 benchmark. For Fino-1, surface roughness shall depend on wind conditions through the Charnock relation, \(z_0 = C_{ch}u_*^2/g\), with \(C_{ch} = 0.0062\), calibrated in neutral conditions for year 2006 in [SR11].
Mesoscale Forcing
You are welcome to use your own mesoscale simulations to feed the meso-micro methodology. However, if you only plan to run microscale simulations, for consistency you should use the mesoscale input forcing provided herein.
Mesoscale input forcing in terms of mesoscale tendencies is provided for the sites following the same methodology of the GABSL3 benchmark case and described in [SRCK17]. For each site, a reference WRF configuration based on one-way telescoping nests at 27, 9 and 3 km horizontal resolution, all of them with grid dimensions of 61x61 and 61 vertical levels up to 5000 Pa. The yearly period is integrated based on two-day runs with an additional day for spin-up. Simulations are initialized at 12UTC using ERA-Interim reanalysis data. The U.S. Geological Survey (USGS) land-use surface data, that comes by default with the WRF model, is used together with the unified Noah land-surface model to define the boundary conditions at the surface. Other physical parameterizations used are: the rapid radiative transfer model (RRTM), the Dudhia radiation scheme and the Yonsei University (YSU) first-order PBL scheme.
A NetCDF file for mesoscale data is provided with the following information:
Site coordinates and Coriolis parameter
Time-height 2D arrays of velocity components (U, V, W) and potential temperature (Th)
Time-height 2D arrays of mesoscale forcings (tendencies): geostrophic wind (Ug, Vg), advective wind (Uadv, Vadv) and advective potential temperature (Thadv)
Time array of surface-layer quantities: friction velocity (ust), kinematic heat flux (wt), 2-m temperature (T2), skin temperature (TSK), surface pressure (Psfc)
Units, dimensions and variables description are all provided in the NetCDF file. Momentum tendencies are provided in [m s-1] and should be multiplied by the Coriolis parameter (\(f_c = 0.00115 s^{-1}\)) to obtain appropriate forces in [m s-2]. For convenience, we have omitted information about humidity since the assumption of dry-atmosphere is typically adopted by wind energy flow models.
Reference Power Curve
The NREL 5 MW reference power curve will be used to evaluate AEP [JBMS09]. A text file is provided with other additional wind speed relationships computed by NREL.
One year of observations from two tall masts are provided in NetCDF format:
Cabauw: 200-m mast, year 2006 (51.971ºN, 4.927ºE)
Fino-1: 100-m mast, year 2006 (54.0143ºN, 6.5933ºE)
The following quantities of interest will be evaluated at the three sites:
Horizontal wind speed (S) and direction (WD) distributions at a reference hub-height of 80 m.
AEP_gross (p50, p90) at 80 m using the NREL 5MW reference power curve.
Velocity and turbulence intensity profiles for 16 wind direction sectors and three stability classes.
Stability will be characterized based on the local Obukhov length L using the stability parameter z/L , where L is obtained from sonic anemometer measurements at 3 m in Cabauw and at 40 m in Fino-1. Stability classes are defined as follows:
Consistent with the philosophy of the challenge, each participant should develop a plan to span the accuracy vs cost figure. For instance:
A WRF modeler could run yearly simulations starting from the 3-nest configuration of the reference set-up (or a different one) and add other 3 nests switching to LES down to resolutions of the order of 100 m and provide 6 results, one from each nest, for the 3 sites.
A CFD modeler may vary the number of simulations included in the assessment and/or decide to increase resolution or switch to a higher-fidelity turbulence model when switching from planning to design phase.
For each AEP assessment you should provide the cost in cpu-hr.
Also adopting the end-user perspective, the simulations may consider how to best use the onsite measurements to calibrate their model-chain to the reference mast. This is equivalent to a conventional micrositing process in the design phase of ensuring that self-prediction at the reference site is free of bias before extrapolating horizontally or vertically to other target prediction sites. Introducing calibration is clear way of distinguishing between planning and design phase in the accuracy vs cost figure although the cpu-time may be roughly the same.
For consistency with the GABLS3 benchmark, microscale models using Sogachev et al. (2012) [SKL12] k-ε turbulence model shall use this set of constants: κ = 0.4, \(C_{ε1} = 1.52\), \(C_{ε2} = 1.833\), \(σ_k = 2.95\), \(σ_ε = 2.95\) and \(C_μ = 0.03\) [4].
Data should be provided in a single NetCDF file per site, as described in the python template based on the reference WRF simulation. Following the example above, the WRF-LES approach should provide 6x2 = 12 files.
Output quantities and dimmensions:
dimmensions: time (t), height above ground (z), zflux height of surface-layer quantities.
time-height: U, V velocity components, potential temperature (Th), turbulent kinetic energy (TKE).
time: friction velocity (us), kinematic heat flux (wt) and/or Obukhov length (L) at zflux height and 2-m temperature (T2).
To homogenize the output data please consider these indications:
Time series should be provided based on 10-min or 1-hr averages.
Vertical profiles should be provided at the simulation levels.
A python script is provided to help you figure out the output format. Please respect the naming convention for variables to allow automatic post-processing.
Time series: please follow the same format as the input .nc file, mean profiles will be generated in the post-processing.
J. Jonkman, S. Butterfield, W. Musial, and G. Scott. Definition of a 5-MW Reference Wind Turbine for Offshore System Development. Technical Report NREL/TP-500-38060, National Renewable Energy Lab. (NREL), Golden, CO (United States), February 2009. URL: https://www.osti.gov/biblio/947422 (visited on 2020-11-02), doi:10.2172/947422.
J. Sanz Rodrigo, D. Allaerts, M. Avila, J. Barcons, D. Cavar, RA Chávez Arroyo, M. Churchfield, B. Kosovic, J. K. Lundquist, J. Meyers, D. Muñoz Esparza, JMLM Palma, J. M. Tomaszewski, N. Troldborg, MP van der Laan, and C. Veiga Rodrigues. Results of the GABLS3 diurnal-cycle benchmark for wind energy applications. J. Phys.: Conf. Ser., 854:012037, May 2017. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F854%2F1%2F012037 (visited on 2020-07-22), doi:10.1088/1742-6596/854/1/012037.
J. Sanz Rodrigo, R. Chávez Arroyo, P. Gancarski, F. Borbón Guillén, M. Avila, J. Barcons, A. Folch, D. Cavar, D. Allaerts, J. Meyers, and A. Dutrieux. Comparing Meso-Micro Methodologies for Annual Wind Resource Assessment and Turbine Siting at Cabauw. J. Phys.: Conf. Ser., 1037:072030, June 2018. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F1037%2F7%2F072030 (visited on 2020-07-22), doi:10.1088/1742-6596/1037/7/072030.
Javier Sanz Rodrigo. Flux-profile characterization of the offshore ABL for the parameterization of CFD models. November 2011. Publisher: Zenodo. URL: https://zenodo.org/record/4195067 (visited on 2020-11-02), doi:10.5281/zenodo.4195067.
Javier Sanz Rodrigo, Matthew Churchfield, and Branko Kosovic. A methodology for the design and testing of atmospheric boundary layer models for wind energy applications. Wind Energy Science, 2(1):35–54, February 2017. Publisher: Copernicus GmbH. URL: https://wes.copernicus.org/articles/2/35/2017/ (visited on 2020-10-14), doi:https://doi.org/10.5194/wes-2-35-2017.
Andrey Sogachev, Mark Kelly, and Monique Y. Leclerc. Consistent Two-Equation Closure Modelling for Atmospheric Research: Buoyancy and Vegetation Implementations. Boundary-Layer Meteorol, 145(2):307–327, November 2012. URL: https://doi.org/10.1007/s10546-012-9726-5 (visited on 2020-10-15), doi:10.1007/s10546-012-9726-5.
Johan Arnqvist, H. Olivares-Espinosa, and S. Ivanell. Investigation of Turbulence Accuracy When Modeling Wind in Realistic Forests Using LES. In Ramis Örlü, Alessandro Talamelli, Joachim Peinke, and Martin Oberlack, editors, Progress in Turbulence VIII, Springer Proceedings in Physics, 291–296. Cham, 2019. Springer International Publishing. doi:10.1007/978-3-030-22196-6_46.
A. Bechmann, N. N. Sørensen, J. Berg, J. Mann, and P.-E. Réthoré. The Bolund Experiment, Part II: Blind Comparison of Microscale Flow Models. Boundary-Layer Meteorol, 141(2):245, August 2011. URL: https://doi.org/10.1007/s10546-011-9637-x (visited on 2020-07-23), doi:10.1007/s10546-011-9637-x.
J. Berg, J. Mann, A. Bechmann, M. S. Courtney, and H. E. Jørgensen. The Bolund Experiment, Part I: Flow Over a Steep, Three-Dimensional Hill. Boundary-Layer Meteorol, 141(2):219, July 2011. URL: https://doi.org/10.1007/s10546-011-9636-y (visited on 2020-07-23), doi:10.1007/s10546-011-9636-y.
Fred C. Bosveld, Peter Baas, Erik van Meijgaard, Evert I. F. de Bruijn, Gert-Jan Steeneveld, and Albert A. M. Holtslag. The Third GABLS Intercomparison Case for Evaluation Studies of Boundary-Layer Models. Part A: Case Selection and Set-Up. Boundary-Layer Meteorol, 152(2):133–156, August 2014. URL: https://doi.org/10.1007/s10546-014-9917-3 (visited on 2020-10-06), doi:10.1007/s10546-014-9917-3.
Louis-Étienne Boudreault, Sylvain Dupont, Andreas Bechmann, and Ebba Dellwik. How Forest Inhomogeneities Affect the Edge Flow. Boundary-Layer Meteorol, 162(3):375–400, March 2017. URL: https://doi.org/10.1007/s10546-016-0202-5 (visited on 2020-10-23), doi:10.1007/s10546-016-0202-5.
Y. Brunet, J. J. Finnigan, and M. R. Raupach. A wind tunnel study of air flow in waving wheat: Single-point velocity statistics. Boundary-Layer Meteorol, 70(1):95–132, July 1994. URL: https://doi.org/10.1007/BF00712525 (visited on 2020-10-23), doi:10.1007/BF00712525.
Elena Cantero, Fernando Borbón Guillén, Javier Sanz Rodrigo, Pedro Santos, Jakob Mann, Nikola Vasiljević, Michael Courtney, Daniel Martínez Villagrasa, Belén Martí, and Joan Cuxart. Alaiz Experiment (ALEX17): Campaign and Data Report. Technical Report, Zenodo, May 2019. Publisher: Zenodo. URL: https://zenodo.org/record/3187482 (visited on 2020-07-22), doi:10.5281/zenodo.3187482.
Andrew Clifton, Aaron Smith, and Michael Fields. Wind Plant Preconstruction Energy Estimates. Current Practice and Opportunities. Technical Report NREL/TP-5000-64735, National Renewable Energy Lab. (NREL), Golden, CO (United States), April 2016. URL: https://www.osti.gov/biblio/1248798 (visited on 2020-11-03), doi:10.2172/1248798.
J. Cuxart, A. A. M. Holtslag, R. J. Beare, E. Bazile, A. Beljaars, A. Cheng, L. Conangla, M. Ek, F. Freedman, R. Hamdi, A. Kerstein, H. Kitagawa, G. Lenderink, D. Lewellen, J. Mailhot, T. Mauritsen, V. Perov, G. Schayes, G-J. Steeneveld, G. Svensson, P. Taylor, W. Weng, S. Wunsch, and K-M. Xu. Single-Column Model Intercomparison for a Stably Stratified Atmospheric Boundary Layer. Boundary-Layer Meteorol, 118(2):273–303, February 2006. URL: https://doi.org/10.1007/s10546-005-3780-1 (visited on 2020-10-06), doi:10.1007/s10546-005-3780-1.
Ebba Dellwik, Ferhat Bingöl, and Jakob Mann. Flow distortion at a dense forest edge. Quarterly Journal of the Royal Meteorological Society, 140(679):676–686, 2014. _eprint: https://rmets.onlinelibrary.wiley.com/doi/pdf/10.1002/qj.2155. URL: https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.2155 (visited on 2020-10-23), doi:10.1002/qj.2155.
Martin Dörenkämper, Bjarke T. Olsen, Björn Witha, Andrea N. Hahmann, Neil N. Davis, Jordi Barcons, Yasemin Ezber, Elena García-Bustamante, J. Fidel González-Rouco, Jorge Navarro, Mariano Sastre-Marugán, Tija Sīle, Wilke Trei, Mark Žagar, Jake Badger, Julia Gottschall, Javier Sanz Rodrigo, and Jakob Mann. The Making of the New European Wind Atlas – Part 2: Production and evaluation. Geoscientific Model Development, 13(10):5079–5102, October 2020. Publisher: Copernicus GmbH. URL: https://gmd.copernicus.org/articles/13/5079/2020/ (visited on 2020-10-27), doi:https://doi.org/10.5194/gmd-13-5079-2020.
Martin Dörenkämper, Michael Optis, Adam Monahan, and Gerald Steinfeld. On the Offshore Advection of Boundary-Layer Structures and the Influence on Offshore Wind Conditions. Boundary-Layer Meteorol, 155(3):459–482, June 2015. URL: https://doi.org/10.1007/s10546-015-0008-x (visited on 2020-10-22), doi:10.1007/s10546-015-0008-x.
Martin Dörenkämper, Björn Witha, and Julia Gottschall. Large-Eddy and Reynolds-Averaged-Navier Stokes simulations of the Kassel forested hill - Rödeser Berg (Deliverable D2.13). Technical Report, Zenodo, October 2019. Publisher: Zenodo. URL: https://zenodo.org/record/3519169 (visited on 2020-07-22), doi:10.5281/zenodo.3519169.
James Edson, Timothy Crawford, Jerry Crescenti, Tom Farrar, Nelson Frew, Greg Gerbi, Costas Helmis, Tihomir Hristov, Djamal Khelif, Andrew Jessup, Haf Jonsson, Ming Li, Larry Mahrt, Wade McGillis, Albert Plueddemann, Lian Shen, Eric Skyllingstad, Tim Stanton, Peter Sullivan, Jielun Sun, John Trowbridge, Dean Vickers, Shouping Wang, Qing Wang, Robert Weller, John Wilkin, Albert J. Williams, D. K. P. Yue, and Chris Zappa. The Coupled Boundary Layers and Air–Sea Transfer Experiment in Low Winds. Bull. Amer. Meteor. Soc., 88(3):341–356, March 2007. Publisher: American Meteorological Society. URL: https://journals.ametsoc.org/bams/article/88/3/341/59046/The-Coupled-Boundary-Layers-and-Air-Sea-Transfer (visited on 2020-10-22), doi:10.1175/BAMS-88-3-341.
H. J. S. Fernando, J. Mann, J. M. L. M. Palma, J. K. Lundquist, R. J. Barthelmie, M. Belo-Pereira, W. O. J. Brown, F. K. Chow, T. Gerz, C. M. Hocut, P. M. Klein, L. S. Leo, J. C. Matos, S. P. Oncley, S. C. Pryor, L. Bariteau, T. M. Bell, N. Bodini, M. B. Carney, M. S. Courtney, E. D. Creegan, R. Dimitrova, S. Gomes, M. Hagen, J. O. Hyde, S. Kigle, R. Krishnamurthy, J. C. Lopes, L. Mazzaro, J. M. T. Neher, R. Menke, P. Murphy, L. Oswald, S. Otarola-Bustos, A. K. Pattantyus, C. Veiga Rodrigues, A. Schady, N. Sirin, S. Spuler, E. Svensson, J. Tomaszewski, D. D. Turner, L. van Veen, N. Vasiljević, D. Vassallo, S. Voss, N. Wildmann, and Y. Wang. The Perdigão: Peering into Microscale Details of Mountain Winds. Bull. Amer. Meteor. Soc., 100(5):799–819, May 2019. Publisher: American Meteorological Society. URL: https://journals.ametsoc.org/bams/article/100/5/799/344800/The-Perdigao-Peering-into-Microscale-Details-of (visited on 2020-07-22), doi:10.1175/BAMS-D-17-0227.1.
J. J. Finnigan, M. R. Raupach, E. F. Bradley, and G. K. Aldis. A wind tunnel study of turbulent flow over a two-dimensional ridge. Boundary-Layer Meteorol, 50(1):277–317, March 1990. URL: https://doi.org/10.1007/BF00120527 (visited on 2020-10-26), doi:10.1007/BF00120527.
Rogier Floors, Alfredo Peña, Guillaume Lea, Nikola Vasiljević, Elliot Simon, and Michael Courtney. The RUNE Experiment—A Database of Remote-Sensing Observations of Near-Shore Winds. Remote Sensing, 8(11):884, November 2016. Number: 11 Publisher: Multidisciplinary Digital Publishing Institute. URL: https://www.mdpi.com/2072-4292/8/11/884 (visited on 2020-07-22), doi:10.3390/rs8110884.
Julia Gottschall, Eleonora Catalano, Martin Dörenkämper, and Björn Witha. The NEWA Ferry Lidar Experiment: Measuring Mesoscale Winds in the Southern Baltic Sea. Remote Sensing, 10(10):1620, October 2018. Number: 10 Publisher: Multidisciplinary Digital Publishing Institute. URL: https://www.mdpi.com/2072-4292/10/10/1620 (visited on 2020-07-22), doi:10.3390/rs10101620.
Andrea N. Hahmann, Tija Sile, Björn Witha, Neil N. Davis, Martin Dörenkämper, Yasemin Ezber, Elena García-Bustamante, J. Fidel González Rouco, Jorge Navarro, Bjarke T. Olsen, and Stefan Söderberg. The Making of the New European Wind Atlas, Part 1: Model Sensitivity. Geoscientific Model Development Discussions, pages 1–33, March 2020. Publisher: Copernicus GmbH. URL: https://gmd.copernicus.org/preprints/gmd-2019-349/ (visited on 2020-07-22), doi:https://doi.org/10.5194/gmd-2019-349.
Charlotte B. Hasager, Andrea N. Hahmann, Tobias Ahsbahs, Ioanna Karagali, Tija Sile, Merete Badger, and Jakob Mann. Europe’s offshore winds assessed with synthetic aperture radar, ASCAT and WRF. Wind Energy Science, 5(1):375–390, March 2020. Publisher: Copernicus GmbH. URL: https://wes.copernicus.org/articles/5/375/2020/ (visited on 2020-07-22), doi:https://doi.org/10.5194/wes-5-375-2020.
A. a. M. Holtslag, G. Svensson, P. Baas, S. Basu, B. Beare, A. C. M. Beljaars, F. C. Bosveld, J. Cuxart, J. Lindvall, G. J. Steeneveld, M. Tjernström, and B. J. H. Van De Wiel. Stable Atmospheric Boundary Layers and Diurnal Cycles: Challenges for Weather and Climate Models. Bull. Amer. Meteor. Soc., 94(11):1691–1706, November 2013. Publisher: American Meteorological Society. URL: https://journals.ametsoc.org/bams/article/94/11/1691/60301/Stable-Atmospheric-Boundary-Layers-and-Diurnal (visited on 2020-10-14), doi:10.1175/BAMS-D-11-00187.1.
IEC-TC88. IEC 61400-12-1 Wind turbines - Part 12:1: Power performance measurements of electricity producing wind turbines. Standard IEC 61400-12-1:2017 ed. 2.0, International Electrotechnical Commission, Geneva, CH, September 2017. URL: https://webstore.iec.ch/publication/26603 (visited on 2020-10-27).
IEC-TC88. IEC 61400-1 Wind energy generation systems - Part 1: Design requirements. Standard IEC 61400-1:2019 ed. 4.0, International Electrotechnical Commission, Geneva, CH, February 2019. URL: https://webstore.iec.ch/publication/26423 (visited on 2020-11-03).
IEC-TC88. IEC 61400-3-1 Wind energy generation systems - Part 3-1: Design requirements for fixed offshore wind turbines. Standard IEC 61400-3-1:2019 ed. 1.0, International Electrotechnical Commission, Geneva, CH, April 2019. URL: https://webstore.iec.ch/publication/29360 (visited on 2020-11-03).
Stefan Ivanell, Johan Arnqvist, Matias Avila, Dalibor Cavar, Roberto Aurelio Chavez-Arroyo, Hugo Olivares-Espinosa, Carlos Peralta, Jamal Adib, and Björn Witha. Micro-scale model comparison (benchmark) at the moderately complex forested site Ryningsnäs. Wind Energy Science, 3(2):929–946, December 2018. Publisher: Copernicus GmbH. URL: https://wes.copernicus.org/articles/3/929/2018/ (visited on 2020-10-23), doi:https://doi.org/10.5194/wes-3-929-2018.
Kaimal J.C. and Finnigan J.J. Atmospheric Boundary Layer Flows. Their Structure and Measurement. Oxford University Press, New York, 1994. ISBN 0-19-506239-6.
H. G. Kim and V. C. Patel. Test Of Turbulence Models For Wind Flow Over Terrain With Separation And Recirculation. Boundary-Layer Meteorology, 94(1):5–21, January 2000. URL: https://doi.org/10.1023/A:1002450414410 (visited on 2020-07-23), doi:10.1023/A:1002450414410.
Xiaoli Guo Larsén, Jianting Du, Rodolfo Bolaños, Marc Imberger, Mark C. Kelly, Merete Badger, and Søren Larsen. Estimation of offshore extreme wind from wind-wave coupled modeling. Wind Energy, 22(8):1043–1057, 2019. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/we.2339. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/we.2339 (visited on 2020-10-22), doi:10.1002/we.2339.
J. Mann, N. Angelou, J. Arnqvist, D. Callies, E. Cantero, R. Chávez Arroyo, M. Courtney, J. Cuxart, E. Dellwik, J. Gottschall, S. Ivanell, P. Kühn, G. Lea, J. C. Matos, J. M. L. M. Palma, L. Pauscher, A. Peña, J. Sanz Rodrigo, S. Söderberg, N. Vasiljevic, and C. Veiga Rodrigues. Complex terrain experiments in the New European Wind Atlas. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 375(2091):20160101, April 2017. Publisher: Royal Society. URL: https://royalsocietypublishing.org/doi/full/10.1098/rsta.2016.0101 (visited on 2020-07-22), doi:10.1098/rsta.2016.0101.
A S Monin and A M Obukhov. Basic laws of turbulent mixing in the surface layer of the atmosphere. Contrib. Geophys. Inst. Acad. Sci. USSR, pages 163–187, 1954.
J. Sanz Rodrigo, R. Chávez Arroyo, P. Gancarski, F. Borbón Guillén, M. Avila, J. Barcons, A. Folch, D. Cavar, D. Allaerts, J. Meyers, and A. Dutrieux. Comparing Meso-Micro Methodologies for Annual Wind Resource Assessment and Turbine Siting at Cabauw. J. Phys.: Conf. Ser., 1037:072030, June 2018. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F1037%2F7%2F072030 (visited on 2020-07-22), doi:10.1088/1742-6596/1037/7/072030.
Javier Sanz Rodrigo, Roberto Aurelio Chávez Arroyo, Patrick Moriarty, Matthew Churchfield, Branko Kosović, Pierre-Elouan Réthoré, Kurt Schaldemose Hansen, Andrea Hahmann, Jeffrey D. Mirocha, and Daran Rife. Mesoscale to microscale wind farm flow modeling and evaluation. WIREs Energy and Environment, 6(2):e214, 2017. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/wene.214. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/wene.214 (visited on 2020-07-22), doi:10.1002/wene.214.
Javier Sanz Rodrigo, Roberto Aurelio Chávez Arroyo, Björn Witha, Martin Dörenkämper, Julia Gottschall, Matias Avila, Johan Arnqvist, Andrea Hahmann, and Tija Sile. The New European Wind Atlas Model Chain. J. Phys.: Conf. Ser., 1452:012087, January 2020. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F1452%2F1%2F012087 (visited on 2020-07-22), doi:10.1088/1742-6596/1452/1/012087.
Javier Sanz Rodrigo, Pawel Gancarski, Roberto Chavez Arroyo, Patrick Moriarty, Matthew Chuchfield, Jonathan W. Naughton, Kurt S. Hansen, Ewan Machefaux, Tilman Koblitz, Eoghan Maguire, Francesco Castellani, Ludovico Terzi, Simon-Philippe Breton, Yuko Ueda, John Prospathopoulos, Gregory S. Oxley, Carlos Peralta, Xiadong Zhang, and Björn Witha. IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 1: Flow-over-terrain models. J. Phys.: Conf. Ser., 524:012105, June 2014. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F524%2F1%2F012105 (visited on 2020-10-14), doi:10.1088/1742-6596/524/1/012105.
A. N. Ross and S. B. Vosper. Neutral turbulent flow over forested hills. Quarterly Journal of the Royal Meteorological Society, 131(609):1841–1862, 2005. _eprint: https://rmets.onlinelibrary.wiley.com/doi/pdf/10.1256/qj.04.129. URL: https://rmets.onlinelibrary.wiley.com/doi/abs/10.1256/qj.04.129 (visited on 2020-10-26), doi:10.1256/qj.04.129.
A.N. Ross, S. Arnold, S.B. Vosper, S. D. Mobbs, N. Dixon, and A. G. Robins. A comparison of wind-tunnel experiments and numerical simulations of neutral and stratified flow over a hill. Boundary-Layer Meteorol, 113(3):427–459, December 2004. URL: https://doi.org/10.1007/s10546-004-0490-z (visited on 2020-10-26), doi:10.1007/s10546-004-0490-z.
F. Gonzalez Rouco, E. García Bustamante, A. N. Hahmann, I. Karagili, J. Navarro, B. Tobias Olsen, T. Sïle, and B. Witha. Report on uncertainty quantification (Deliverable D4.4). Technical Report, Zenodo, August 2019. URL: https://zenodo.org/record/3382572 (visited on 2020-07-22), doi:10.5281/zenodo.3382572.
Pedro Santos, Jakob Mann, Nikola Vasiljević, Elena Cantero, Javier Sanz Rodrigo, Fernando Borbón, Daniel Martínez-Villagrasa, Belén Martí, and Joan Cuxart. The Alaiz Experiment: untangling multi-scale stratified flows over complex terrain. Wind Energy Science Discussions, pages 1–24, July 2020. Publisher: Copernicus GmbH. URL: https://wes.copernicus.org/preprints/wes-2020-89/ (visited on 2020-07-22), doi:https://doi.org/10.5194/wes-2020-89.
Javier Sanz Rodrigo, Matthew Churchfield, and Branko Kosovic. A methodology for the design and testing of atmospheric boundary layer models for wind energy applications. Wind Energy Science, 2(1):35–54, February 2017. Publisher: Copernicus GmbH. URL: https://wes.copernicus.org/articles/2/35/2017/ (visited on 2020-10-14), doi:https://doi.org/10.5194/wes-2-35-2017.
A. Smedman, U. Högström, H. Bergström, A. Rutgersson, K. K. Kahma, and H. Pettersson. A case study of air-sea interaction during swell conditions. Journal of Geophysical Research: Oceans, 104(C11):25833–25851, 1999. _eprint: https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/1999JC900213. URL: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/1999JC900213 (visited on 2020-10-22), doi:10.1029/1999JC900213.
P. Taylor and H. W. Teunissen. Askervein ’82: report on the September/October 1982 experiment to study boundary layer flow over Askervein, South Uist. Technical Report MSRS-83-8, Meteorological Services Research Branch, Atmospheric Environment Service, Downsview, Ontario, Canada, 1983. URL: http://www.yorku.ca/pat/research/Askervein/ASK82.pdf (visited on 2020-10-16).
P. Taylor and H. W. Teunissen. The Askervein Hill Project: report on the September/October 1983, main field experiment. Technical Report MSRS-84-6, Meteorological Services Research Branch, Atmospheric Environment Service, Downsview, Ontario, Canada, 1985. URL: http://www.yorku.ca/pat/research/Askervein/ASK83.pdf (visited on 2020-10-16).
P. A. Taylor and H. W. Teunissen. The Askervein Hill project: Overview and background data. Boundary-Layer Meteorol, 39(1):15–39, April 1987. URL: https://doi.org/10.1007/BF00121863 (visited on 2020-07-23), doi:10.1007/BF00121863.
The Offshore Wind Accelerator (OWA) Wake Modelling Challenge was organized within Wakebench Phase 3 with support from the OWA partners. The results were presented at the Torque-2020 conference.
The OWA Challenge finished with 11 participants submitting results from 15 different wake models. While engineering models can predict the overall array efficiency with a bias of a few percent, this is through compensation of errors at turbine level and between wind climate bins that can be larger than 20%. The assessment is challenged by the lack of undisturbed measurements of the gross power which can show variations of more than 20% across large wind farms under the influence of farm-farm and coastal horizontal wind speed gradients.
The primary objective is to understand the limitations of wake models used by industry for the prediction of array efficiency over a relevant range of operational conditions in the offshore environment. To this end the following individual objectives are agreed with OWA partners:
Evaluate wake modelling and power prediction methods and validate the results with measured data.
Examine the accuracy of specific models, quantify uncertainty bands and highlight modelling trends.
Define an open-access model evaluation methodology that can systematically improve consistency and traceability in the assessment of state-of-the-art wake models for power prediction as more datasets are added.
Blind test pilot: Benchmark participants don’t have access to the validation data, only the benchmark manager. The Anholt case is launched first as a pilot to develop the model evaluation methodology together with the participants as they are engaged. Evaluation scripts are shared with the participants so they can run their own assessment and submit the best prediction.
Blind test multisite: The blind test is extended to the other sites as soon as the input data is available. Participants with engineering models should submit predictions for all the sites and bins. Participants using CFD models can limit the submission to one wind direction sector (indicated below) to limit the computational effort.
Calibration: One benchmark participant from the blind test phase is selected to iterate further with a model using calibrated input data. Since observational data cannot be shared, a sensitivity analysis on model parameters is used as a proxy to analyze model calibration.
Registered participants receive an identification code (userID) which they use to submit their data and identify their results in an anonymous model intercomparison. They are also required to fill in a questionnaire providing details about their simulations to facilitate the assessment of the results as well as their data-sharing position.
The OWA Wake Modeling Challenge is an Offshore Wind Accelerator (OWA) project that aims to improve confidence in wake models in the prediction of array efficiency. Model developers and end-users were invited to participate in a benchmarking exercise along a suite of validation datasets that spans a wide range of operational conditions in terms of wind farm topology and wind climate characteristics.
The validation strategy focuses on large offshore wind farms and turbines, of up to 6 MW, and addresses limitations of engineering wake models, traditionally developed under the assumption of surface-layer inflow conditions (horizontally homogeneous, steady-state). In effect, large wind farms are subject to heterogeneous inflow conditions due to coastal gradients, wakes from neighboring wind farm clusters and large-scale mesoscale phenomena. The interaction of the atmospheric boundary-layer (ABL) with the wind farm canopy includes different processes from the upstream blockage of the flow to the generation of an internal boundary-layer that, in very large arrays, results in a fully developed wind farm boundary layer where turbulent mixing between wake effects and the outer ABL is in equilibrium and array efficiency becomes constant (deep-array effect). Under stable conditions the ABL height is compressed to a few hundred meters and a low-level jet forms. In the presence of large wind farms, this low-turbulence regime generates long-lasting wakes and introduces significant blockage in the incoming flow which results in important array efficiency reductions compared to neutral or unstable conditions. Within the wind farm, local acceleration effects near corners or within gaps are opportunities for wind farm design optimization. Altogether, the complex system is the result of a multi-scale process that requires a systematic assessment to understand the relative importance of each phenomenon in the prediction of array efficiency for energy yield assessment and wind farm design.
In the absence of free-stream meteorological observations, a mesoscale-to-wake modelling framework is established whereby mesoscale simulations are provided as input data for the modellers to interpret in connection to their wake models. Then, the power predicted at turbine position i can be defined as:
where \(A_{S,i}\) and \(A_{M,i}\) are correction factors to account for changes in power within the array, due to horizontal wind speed gradients, and mesoscale bias respectively. Assuming power-curve relationships between power and wind speed:
where \(S_i\) is the background mesoscale wind speed and \(S_{obsfree,i}\) is the observed equivalent free-stream wind speed, which is obtained from the observed power \(P_{obsfree,i}\) at wake-free turbines though the inverse of the power curve. Since \(S_{obsfree,i}\) is deduced from the power of the free-stream turbines, we assume that potential effects of upstream wind farm induction are included in \(A_{M,i}\), which would correct for both mesoscale and induction or blockage. According to [BPRT18] wind farm induction effects can result in up to 4% overestimation of array efficiency in engineering wake model when they operate at the maximum thrust coefficient. Hence, it is worth removing this effect in the assessment of array efficiency so we can be sure that it is only related to wake losses.
Mesoscale simulations using the NEWA WRF set-up [HSW+20][DOW+20] are produced to generate background wind conditions for wake models that are free of (microscale) site effects. The WRF set-up consists on three one-way nested domains of 27, 9 and 3 km resolution are configured centred at the wind farm centroid (Table 3). The vertical grid has 61 terrain-following (sigma) levels, with 10 levels covering the first 200 meters, more specifically at: 6, 22, 40, 56, 73, 90, 113, 140, 179 and 205 meters. High resolution topography (SRTM 90m) and updated land use categories (Corine Land Cover 2018), together with the Noah land-surface model are used to define the boundary conditions at the surface. The physical parameterizations are: Mellor–Yamada–Nakanishi Niino 2.5-level planetary boundary-layer scheme (MYNN), WRF Single-Moment 5-class microphysics scheme, the Rapid Radiative Transfer Model for GCMs shortwave and longwave radiation schemes and the Kain-Fritsch cumulus scheme in the outermost domains 1 and 2. The simulation is driven by input data from ERA-5 in blocks of 5 days with additional spin-up time of 24 hours.
We use the centroid of the mesoscale simulation to define reference wind conditions in terms of hub-height interpolated wind speed and direction and surface-layer stability, defined by \(z/L\) parameter where z = 10 m and L is the Obukhov length computed by WRF. Reference wind conditions are obtained by horizontal averaging those at the wind turbine sites.
Neighbouring wind farms have also been simulated using the Fitch wind farm parameterization available in WRF. The innermost nest in the simulations has a horizontal resolution of 3 km. Following the same approach that was adopted in the EERA-DTOC project [SPE+15], two sets of input data are generated:
Control (ctrl): Free of wake effects, to characterize the background wind resource.
Wakes (wakes): Includes neighbor wind farms but not the target wind farm, to characterize inflow conditions for microscale wake modeling.
For each mesoscale simulation two sets of data are derived:
_WindTurbines.nc: time series of wind conditions interpolated at each turbine site.
_ref.nc: horizontal average of _WindTurbines.nc to define reference wind conditions as if it was a virtual mast at the wind farm centroid.
Turbine layout coordinates are provided in the _layout.csv file. Generic power and thrust curves have been kindly provided by EMD.
All the input data is provided to registered participants through a b2drop input folder.
Supervisory control and data acquisition (SCADA) operational data is used to perform the validation for a period of at least one year (Table 3). A quality control process has been carried out to produce a “clean” dataset that only includes situations where a turbine is working in nominal conditions, i.e. whose power output is close to the value predicted by the theoretical power curve and, therefore, corresponds to the operational conditions simulated by wake models in the pre-construction phase.
A machine learning technique is used for gap filling to recover time instances when only a few turbines are working in non-nominal conditions. In these situations, a regression algorithm trained on clean data from neighbour turbines predicts corrected data for the missing turbines to obtain a complete dataset. The effect on the overall array efficiency assessment is minor compared to the benefits of obtaining a validation dataset that is more statistically significant.
As a result, the quality-control corrected data consist of hourly timestamps of power output and nacelle wind direction with all the turbines working in nominal conditions.
Then, validation data is defined in terms of sector-wise and stability-wise ensemble averages for 30º wind direction sectors and a 9±1 m/s velocity bin, when the thrust coefficient is at its maximum resulting in stronger wake effects. The data is subdivided into three stability classes according to the reference stability \(z/L\) parameter simulated by WRF:
Unstable (u): -0.2 < \(z/L\) < -0.02
Neutral (n): -0.02 < \(z/L\) < 0.02
Stable (s): 0.02 < \(z/L\) < 0.2
Mesoscale and SCADA hourly data is synchronized and flagged to filter out registers in non-nominal conditions that will not participate in the validation.
A fundamental question that the modeler has is how large does the microscale domain need to be. In a single wind farm approach we shall simulate the target wind farm at microscale with no explicit modeling of the shore or neighbor wind farms, since we trust that mesoscale simulations already include these effects. Then, you can assume horizontal homogeneity in the inflow conditions by using the reference virtual mast to define the inflow conditions for all the turbines (WindFarm_Wakes_ref.nc). Alternatively, you can use heterogeneous wind conditions at each wind turbine (WindFarm_Wakes_WindTurbines.nc, i.e. hub-height interpolated data).
The alternative, the cluster approach, will simulate the shore and wind farms situated close enough in the same microscale simulation using ctrl input data.
We suggest that all participants submit results following the single wind farm approach to come up with a consistent database of results based on the most simple and efficient approach. In the case of Rødsand 2 and Ormonde we will use both methods to understand the impact when the separation between wind farms is small.
The ultimate goal is to analyse ensemble-averaged statistics of array efficiency \(\eta\), at individual turbine level and for the whole wind farm, defined as:
where \(P_i\) and \(S_i\) are the power and free-stream wind speed at turbine position i. Note that the gross power is defined in terms of the theoretical power curve, P(Si), at each turbine position as if it were operating in isolation. This is in contrast to previous validation studies that assumed uniform free-stream conditions for the entire wind farm.
Since the intended use of the models focuses on energy yield assessment, we are interested in the validation of long-term averaged conditions, categorized by wind direction, wind speed and stability classes. Hence, the validation metrics will be based on bin-averaged conditions and not on the analysis of time series in connection to specific weather conditions.
Participants can submit their results as ensemble-averaged quantities or as time-series. Only formatted data following the benchmark guide should be submitted. You are encouraged to use the model evaluation script to test compatibility and self-assess your results by comparing with other model results that may be available. Please attach documentation that will help interpret your simulations and your self-assessment.
Input and output data is facilitated through shared folders in b2drop.
James Bleeg, Mark Purcell, Renzo Ruisi, and Elizabeth Traiger. Wind Farm Blockage and the Consequences of Neglecting Its Impact on Energy Production. Energies, 11(6):1609, June 2018. Number: 6 Publisher: Multidisciplinary Digital Publishing Institute. URL: https://www.mdpi.com/1996-1073/11/6/1609 (visited on 2020-12-15), doi:10.3390/en11061609.
Martin Dörenkämper, Bjarke T. Olsen, Björn Witha, Andrea N. Hahmann, Neil N. Davis, Jordi Barcons, Yasemin Ezber, Elena García-Bustamante, J. Fidel González-Rouco, Jorge Navarro, Mariano Sastre-Marugán, Tija Sīle, Wilke Trei, Mark Žagar, Jake Badger, Julia Gottschall, Javier Sanz Rodrigo, and Jakob Mann. The Making of the New European Wind Atlas – Part 2: Production and evaluation. Geoscientific Model Development, 13(10):5079–5102, October 2020. Publisher: Copernicus GmbH. URL: https://gmd.copernicus.org/articles/13/5079/2020/ (visited on 2020-10-27), doi:https://doi.org/10.5194/gmd-13-5079-2020.
Andrea N. Hahmann, Tija Sile, Björn Witha, Neil N. Davis, Martin Dörenkämper, Yasemin Ezber, Elena García-Bustamante, J. Fidel González Rouco, Jorge Navarro, Bjarke T. Olsen, and Stefan Söderberg. The Making of the New European Wind Atlas, Part 1: Model Sensitivity. Geoscientific Model Development Discussions, pages 1–33, March 2020. Publisher: Copernicus GmbH. URL: https://gmd.copernicus.org/preprints/gmd-2019-349/ (visited on 2020-07-22), doi:https://doi.org/10.5194/gmd-2019-349.
J. Sanz Rodrigo, F. Borbón Guillén, P. M. Fernandes Correia, B. García Hevia, W. Schlez, S. Schmidt, S. Basu, B. Li, P. Nielsen, M Cathelain, Dall’Ozzo C., L. Grignon, and D. Pullinger. OWA Wake Modelling Challenge. June 2020. URL: https://zenodo.org/record/4321054 (visited on 2020-12-14), doi:10.5281/zenodo.4321054.
Javier Sanz Rodrigo, Fernando Borbón Guillén, Pedro M. Fernandes Correia, Bibiana García Hevia, Wolfgang Schlez, Sascha Schmidt, Sukanta Basu, Bowen Li, Per Nielsen, Marie Cathelain, Cédric Dall’Ozzo, Laure Grignon, and David Pullinger. Validation of Meso-Wake Models for Array Efficiency Prediction Using Operational Data from Five Offshore Wind Farms. J. Phys.: Conf. Ser., 1618:062044, September 2020. Publisher: IOP Publishing. URL: https://doi.org/10.1088%2F1742-6596%2F1618%2F6%2F062044 (visited on 2020-12-14), doi:10.1088/1742-6596/1618/6/062044.
Javier Sanz Rodrigo, Fernando Borbón Guillén, Pedro M. Fernandes Correia, Bibiana García Hevia, Wolfgang Schlez, Sascha Schmidt, Sukanta Basu, Bowen Li, Per Nielsen, Marie Cathelain, Cédric Dall’Ozzo, Laure Grignon, and David Pullinger. OWA Wake Modelling Challenge Dataset. March 2020. type: dataset. URL: https://zenodo.org/record/3715198 (visited on 2020-12-14), doi:10.5281/zenodo.3715198.
G. Schepers, A. Peña, A. Ely, A. Palomares, A. Attiya, G. Sioros, G. Giebel, H. Svendsen, I. Bastigkelt, I. Karagali, I. Moya, O. Anaya Lara, P. Ledesma, P. M. Fernandes Correia, V. Gosta Gomez, and W. He. Eera dtoc calculation of scenarios (Deliverable D5.12). Technical Report, EERA DTOC, June 2015. URL: http://www.eera-dtoc.eu/wp-content/uploads/files/D5_12_Scenarios-for-wind-farm-clusters.pdf (visited on 2020-12-15).
The benchmark has been carried out with support from the Offshore Wind Accelerator OWAbench project. We would like to thank Carbon Trust and the OWA Technical Working Group for their support providing funding, operational data and guidance throughout the project. We would like to thank all the benchmark participants for their simulations and in-kind support in fine-tuning the benchmark set-up and evaluation methodology.
