Prediction of Precipitating Weather Systems

The WRF model is being developed as a collaborative effort among the NCAR Mesoscale and Microscale Meteorology Division (MMM), NCEP’s Environmental Modeling Center (EMC), FSL’s Forecast Research Division (FRD), the DoD Air Force Weather Agency (AFWA), the Center for the Analysis and Prediction of Storms (CAPS) at the University of Oklahoma, and the Federal Aviation Administration (FAA), along with the participation of a number of university scientists. Primary funding for MMM participation in WRF is provided by the NSF/USWRP, AFWA, FAA and the DoD High Performance Modernization Office. With this model, we seek to improve the forecast accuracy of significant weather features across scales ranging from cloud to synoptic, with priority emphasis on horizontal grids of 1-10 kilometers. The model will incorporate advanced numerics and data assimilation techniques, multiple relocateable nesting capability and improved physics, particularly for treatment of convection and mesoscale precipitation systems. It will be well suited for a range of applications, from idealized research to operational forecasting, and have flexibility to accommodate future enhancements.

The WRF model has these desirable characteristics: It is designed to be highly modular, and a single source code will be maintained that can be configured for both research and operations. It will be state-of-the-art, transportable, and efficient in a massively parallel computing environment (accommodating vector environments as well). Data assimilation systems and adjoint and tangent linear forms (for 3DVAR analysis and 4DVAR assimilation) will be developed in tandem with the model itself. Numerous physics options will be allowed, thus tapping into the experience of the full modeling community. It will be maintained and supported as a community mesoscale model to facilitate broad use in research, particularly in the university community. Research advances will have a direct path to operations. With these hallmarks, the WRF model is unique in the history of numerical weather prediction in the U.S.

During the past year, the WRF system has advanced substantially, facilitated by real-time experimental forecasting and the community release of WRF for further evaluation and testing. When the WRF model becomes sufficiently mature to be used operationally, it is expected to (1) replace the Meso-Eta model for the operational Threats forecasts at NCEP, (2) replace the MM5 model for operational use by AFWA and (3) take on the function of rapid updating, now served by the RUC model.

Recognizing the research focus within the WRF effort, alternative numerical techniques continue to be explored and adapted to the WRF framework to facilitate the comparative evaluation of their relative accuracy and efficiency in a controlled computational environment. Work has been progressing on three candidate prototype solvers; two of these prototypes are split-explicit Eulerian models based on mass and height vertical coordinates, respectively, while the third is a semi-implicit semi-Lagrangian formulation. Both the mass and height coordinate Eulerian are now available as run-time selectable cores within the WRF model framework.

During the past year, William Skamarock has implemented the Eulerian, split-explicit, flux-form, terrain-following mass coordinate prototype within the WRF model computational framework. The implementation includes 3rd order Runge Kutta (RK3) time integration methods that allow the use of high-order upwind advection operators and do not suffer from the large dispersion errors found in leapfrog schemes. The RK3 scheme, developed by Lou Wicker (NOAA/NSSL) and Skamarock, allows the use of both upwind (odd ordered) and centered (even ordered) high-order flux-divergence operators. It also exhibits low dispersion errors and allows a larger time step when used with either odd or even higher order advection operators. The RK3 scheme has been demonstrated to be robust and, as anticipated, the higher order schemes advection schemes are producing superior solutions at marginal resolution.

Both prototypes have been tested using idealized simulations of a variety of test cases covering a broad range of scales, including simulations of synoptic-scale baroclinic waves in a periodic channel with a 100-km horizontal grid and supercell thunderstorm evolution with a 1-km grid. These simulations and others are providing benchmarks for the WRF prototypes with published solutions from other models, and they demonstrate the robustness and accuracy of the new approaches used in the WRF prototypes. In order to provide an early capability to initialize the mass-coordinate version with real data, David Gill and Jimy Dudhia developed a converter program to interpolate the initial fields from the height coordinate provided by the Standard Initialization Package to the mass-coordinate grid.

Evaluation of idealized simulations has contributed to further improvements in the dynamic-model solver. In conducting mountain wave simulations Oliver Fuhrer (Swiss Federal Institute of Technology, Zurich), encountered artificial disturbances over small-scale terrain in the community release of WRF, as well as in other models. Joseph Klemp and Skamarock demonstrated that these errors are contained in the linear system of equations, and explained their behavior through analytic solutions to the steady-state, finite-difference equations. Their analysis documents that these errors arise if the order of accuracy in computing the metric terms associated with the terrain following coordinates was not the same as the accuracy used for the horizontal advection. The numerics for the metric terms in the WRF code were modified to insure consistency in these calculations. Klemp and Skamarock also used idealized mountain-wave simulation to design and implement a new filter that selectively removes small scale external modes that can arise during startup in the mass-coordinate prototype over regions of significant terrain.

Development of the semi-implicit semi-Lagrangian prototype is being led by Jim Purser (NOAA/NCEP). Purser has developed a package of efficient compact or Pade schemes, and implemented them within the WRF framework. These methods attain a high formal order of accuracy for the spatial operations of differentiation and quadrature, and form an integral part of the high-order, conserving, cascade interpolations used in the grid-to-grid interpolations needed for the semi-Lagrangian calculations. Purser is also developing high order Runge-Kutta time integration methods for semi-implicit solvers, as well as a hybrid vertical coordinate, for the semi-Lagrangian prototype. The solver for this prototype is presently under development and will be evaluated in comparison with the Eulerian prototypes.

Also within the context of the WRF model solvers, Skamarock, Stan Benjamin (NOAA/Forecast Systems Laboratory), Riener Bleck (Los Alamos Laboratory) and Zuwen He (University of Miami) have been developing hybrid coordinate model formulations for the nonhydrostatic compressible equations. The hybrid coordinate takes the form of a terrain-following sigma-like coordinate near the surface and relaxes to an isentropic coordinate (or any other specified coordinate) aloft. Zuwen has developed an initial hybrid approach based on an explicit solution technique that splits the integration of the acoustic modes from the coordinate-surface movement. Successful simulations of baroclinic waves and mountain waves indicate that further testing with convection and nonhydrostatic phenomena is warranted.

The WRF project continues to play a role in the evolution of frameworks and component architectures for high-performance computing in the atmospheric sciences. The WRF-developed Advanced Scientific Framework, in addition to supporting rapid development and deployment of the WRF model itself, has been adapted to WRF and MM5 3DVAR by Al Bourgeois and to other non-WRF models, such as the NOAA/NCEP's non-hydrostatic Eta model (Tom Black, NOAA/NCEP). The WRF software architecture consists of three distinct model layers: a solver layer that is usually written by scientists, a driver layer that is responsible for allocating and deallocating space and controlling the integration sequence and I/O, and a mediation layer that glues these pieces together. It supports a multi-level approach to parallelism adaptable, without change to the source code, to single-processor, shared-memory, distributed-memory and hybrid-parallel systems. The WRF software framework also provides performance portability across micro- and vector-processors. A novel aspect of this modeling system is its use of a data registry. The data registry, designed and implemented by John Michalakes, is the single place where developers list model variables and their characteristics. The WRF project is represented on the NASA-funded Earth System Modeling Framework (ESMF) project to foster reuse and interoperability of software in the geosciences. Work is also underway to leverage developments at DOE, DoD and other institutions to extend WRF software for inter-model coupling.

In order to streamline the handling of I/O throughout the many components of the overall system, Michalakes, Leslie Hart and Jacques Middlecoff (both NOAA/FSL) and Dan McCormick (AFWA) have designed and implemented an I/O Application Program Interface (API) that provides a standard way of specifying and accessing data within the model that is independent of any particular I/O package. For the initial version of WRF, they are using the API to implement the model I/O based on the NetCDF format. Other data formats, such as HDF and GRIB, will be coupled to the I/O API as the code matures. At present, work is continuing to integrate the new I/O interface within the WRF software framework and refine the formats used for the NetCDF output files. Michalakes and Jim Tuccillo (IBM) have developed and implemented the NCEP asynchronous I/O capability called "quilting" into the WRF framework layer responsible for I/O. Quilting designates a number of additional I/O server processes to collect and write model output so that model integration can proceed with minimal interruption due to I/O. Quilting will be part of the WRF 1.2 community release.

The rapid pace of WRF-model development has been greatly facilitated by the modular, hierarchical WRF design. Demonstrating the effectiveness of the plug-compatible WRF model-layer interface and the WRF data registry, Shu-Hua Chen (visitor, AFWA), Jimy Dudhia, Wei Wang, and David Gill, have been able to incorporate numerous physics packages into WRF in remarkably little time. By adhering to the WRF interface specification and coding conventions, the physics packages are automatically interoperable over shared- and distributed-memory parallel computers. The WRF software framework also supports multiple dynamical cores, selectable at run-time. Current options are the two Eulerian dynamic-core prototypes: one a height-based and the other a mass-based vertical-coordinate formulation. The semi-implicit semi-Lagrangian core under development at NOAA/NCEP will be another option when it becomes available.

In addition to interoperability, the WRF software aims at high-performance over a range of computing architectures using a single maintainable source code. WRF is currently ported to and supported on IBM, Compaq, SGI, Sun and Fujitsu systems as well as Linux-clusters (both Intel- and Alpha-based). Evaluation of performance and optimization testing was presented to HPC Asia 2001, an international conference on high-performance computing in September, and this work is continuing. WRF was one of the performance benchmark applications in the recent acquisition of the NCAR Advanced Research Computing System. The WRF real-time forecast system shows good performance and scaling efficiency, running at up to 90 billion floating-point operations per second on the large NSF Terascale Computing System (a 6 Teraflop/second Compaq supercomputer installed late in 2001 at the Pittsburgh Supercomputing Center). The test problem (see Figure 11) is a 12 km resolution 48-hour forecast over the Continental U.S. that captures the development of a strong baroclinic cyclone and a frontal boundary that extends from north to south across the entire U. S. This forecast executes in 10 minutes on 512 of the 3000 processors of the TCS, at a rate of almost 90 Gigaflop/second (not counting I/O time) (see Figure 12).

Jimy Dudhia and Shu-Hua Chen (visitor, AFWA) have continued to collaborate in incorporating a variety of physics options within WRF. The latest packages include a microphysical option, a boundary-layer option, a radiation option from the current Eta model physics packages and a land-surface model very similar to that in the Eta model. Collaborators for these schemes were Tom Black (NCEP/EMC), Fei Chen and Hsiao-Ming Hsu (both NCAR/RAP). S.-H. Chen also generalized the physics interface to work with both the mass and height coordinate versions of the WRF dynamics. These schemes are being tested prior to implementation in the next release of WRF.

The WRF Land Surface Model is being developed by F. Chen, together with scientists at NCEP and AFWA, as part of a project to unify and extend versions of the OSU LSM currently used by NCAR, AFWA and NCEP. David Gill has worked with Dudhia, in collaboration with Brent Shaw and John Smart (both NOAA/FSL), to bring land-surface data into the WRF model to support the LSM. Meanwhile, F. Chen and Hsu, with the help of S.-H. Chen, have implemented the land-surface model into WRF. The land-surface model parameterizes soil moisture, snow cover, skin temperature and vegetation processes. Workshops on land-surface modeling to coordinate this unification effort were held at NCEP in October, 2000, and NCAR in August, 2001.

The development of WRF physics is an ongoing research effort that must rely significantly on community participation. Therefore, a standard physics interface has been designed in order to streamline participation in developing and adapting physics to WRF. In this interface, the model solver calls a generic driver for each class of physics, which in turn calls the specific desired package. Thus, user-developed packages plug into the physics driver through the standard interface, and remain isolated from the model solver.

A major effort undertaken this past year has been the testing of both the height and mass coordinate WRF prototypes in real-time NWP applications. The real-time forecasting experiments are important in at least two aspects. First, they allow the new model to be evaluated under a large number of weather regimes, and synoptic conditions. These forecasts allow the development team to examine the model performance daily and detect any systematic problem. Second, they provide a test bed to test the new model's robustness under various weather conditions. Wei Wang began running the height coordinate model twice daily at NCAR since December, 2000. She began initially with a 30-km horizontal resolution CONUS (Continental U.S.) grid and later a 22-km CONUS grid (a grid similar to that in operational Eta model from NCEP before November, 2001). This prototype is also run daily on a 10-km grid over the Central U. S. The mass coordinate WRF model prototype has been run in real-time configuration since late August, 2001 on a 22-km CONUS grid. All model runs are initialized with Eta model data. Verification of the quantitative precipitation in these forecasts has been provided by Mike Baldwin (NOAA/NSSL/SPC) and compared to forecasts from the NCEP Eta model, an NSSL modified Eta model, the NCAR MM5 model and an NSSL WRF model (Jack Kain, NSSL). These comparisons are displayed on the NSSL Web site, http://www.nssl.noaa.gov/etakf/qpfplots/, and have been very useful for identifying and solving initial problems with the microphysics in real-data applications (see Figure 13).

They have also served as a valuable tool in validating the newer mass dynamical core, and allowed the development team to detect and correct some subtle problems in the mass-version dynamics. The height and mass versions are now producing very comparable forecasts (see Fig. 14). Mike McAtee (AFWA) has also been running real-time WRF forecasts for Air Force theatres and validating results against surface and sounding data. All the real-time WRF forecasts are linked from the WRF Web pages, http://wrf-model.org/REAL_TIME/real_time.html.

One of the primary objectives of the WRF developmental effort is to improve our ability to represent and forecast convective systems in the 6-12 hour time frame. The success of such an effort depends on many factors, including the use of sufficient resolution to represent the convective processes, accurately representing the mesoscale environment of the convective system, and appropriately forecasting the timing and location of significant convective triggering. As an important step in testing WRF’s abilities in this regard, an effort has begun to simulate and forecast significant convective outbreaks. An example is presented here from 11 June 2001. A severe convective system was spawned over western Minnesota early in the afternoon and organized into a large, bow-echo squall line with a dominant cyclonic mesoscale convective vortex in southeastern Wisconsin and northern Illinois that evening. This storm produced widespread wind damage and creating large disruptions in air travel. Real-time 30-km grid forecasts, however, merely indicated the potential for heavy rainfall over an area much broader than the observed event (see Figure 13). Morris Weisman and Wei Wang conducted a 16-hr, 4-km grid simulation with the WRF model initiated at 12 GMT, which forecast the structure, propagation and intensity of this system amazingly well, despite some errors in the timing and location of the initial convective triggering (see Figure 15). Such results offer hope that enhanced resolution can improve the short-term prediction of such significant convective events in meaningful ways.

Stanley Trier and Christopher Davis have begun to develop a suite of test cases over a broad range of meteorological regimes, which both WRF developers and the community can use to better understand the sensitivities of the model. The emphasis in this work has been on selecting observed cases that encompass a wide variety of meteorological regimes, and examining model sensitivities, such as resolution and physical parameterizations (e.g., PBL, cumulus and microphysical schemes) in these cases. These observed cases augment a preexisting set of idealized cases (e.g., supercell thunderstorms, squall lines, 2-D mountain-wave flow). The most detailed examination of model performance and sensitivities for these observed cases has been for simulations of a multi-day episode of organized convection (27-29 May 1998), and a shallow arctic cold front (10-12 December 2000) confined to the locations east of the continental divide. The WRF simulations captured the essence of these meteorologically diverse phenomena and compared favorably with similar simulations using other models (e.g., ETA, MM5). Despite the overall success of these simulations, there were significant errors in important details (in all of the models), including the magnitude of the precipitation in the 27-29 May case and the speed and intensity of the arctic front (particularly near the Continental Divide) on day two of that simulation. The magnitude of such errors were found to be sensitive to model configurations and parameters, such as cumulus schemes (27-29 May) and horizontal resolution (10-12 December). Ongoing and future work comprises an examination of WRF simulations of additional observed cases, including a rapidly deepening midlatitude cyclone, a tropical cyclone and orographically forced precipitation.

In May, 2001, the MM5 3DVAR system was chosen as the starting point for initial data assimilation capabilities of the WRF 3DVAR. Since that time, collaborators at NCEP (Derber, Wu), FSL (Devenyi), AFWA (McAtee) and CAPS (Xue, Gao) have begun work on the inclusion of additional capabilities. As part of these efforts, Wu has added the capability to read observational data files in BUFR format. Barker has modified the grid staggering in the 3DVAR system from the Arakawa B-grid of MM5 to an unstaggered grid, which has chosen for WRF 3DVAR for generality and simplicity. Bourgeois has modified the WRF software framework to accommodate the WRF 3DVAR and has extended the framework’s capabilities to provide parallelism for the 3DVAR code, both for WRF and MM5 applications. All MM5 3DVAR applications have now adopted the WRF 3DVAR system, allowing MMM 3DVAR efforts to concentrate on a single data-assimilation system. The first release of a basic version of the WRF 3DVAR, coupled to the WRF forecast model, is expected around the end of calendar year 2001.

Some of the priority objectives of the WRF project are to make the model and ancillary programs available to the broader research community, to facilitate use of the model in a wide variety of applications, and to solicit participation from the research community in the continuing evolution of the model. Specific tasks within MMM, therefore, include maintenance of up-to-date code, distribution of code, aiding in documentation of the modeling system, maintenance of WRF Web sites and mailing lists, provision of a user support e-mail address, distribution of WRF announcements and organization of WRF workshops and tutorials.

Joseph Klemp, William Skamarock, John Michalakes, Jimy Dudhia, Shu-Hua Chen, David Gill and Wei Wang, in cooperation with WRF developers at NOAA/FSL, released WRF Version 1.0 in December, 2000, followed by Version 1.1 in May, 2001. For these releases, a Web site for registering as a WRF user and downloading code has been provided. The primary supported programs are the model itself and the Standard Initialization. MMM also provides a converter from MM5 input to WRF input and some graphics capabilities with NCL and Vis5d. The user support e-mail address (wrfhelp@ucar.edu or wrfhelp@wrf-model.org) has been established for user questions.

This first release integrates the fully compressible nonhydrostatic equations in scalar-conserving flux form using a time-split small step for acoustic modes. Large time steps utilize the Runge-Kutta techniques discussed above and 2nd to 6th order advection operators can be specified. The vertical coordinate is a terrain following height coordinate that allows variable resolution with height. It is initially a single domain version and contains map-scale factors for conformal projections. The model code is written in standard Fortran 90 and is self-contained. It will run in parallel on both shared-memory and distributed memory platforms. The model can be configured to run either idealized or real data simulations. For idealized simulations, periodic, symmetric or open radiative lateral boundary conditions are available. For real-data cases, initial fields are interpolated from GRIB or MM5 files, and model physics can be selected from the above-mentioned options. Lateral boundary conditions are specified and merged to the interior with a relaxation zone.

During FY01, 375 users registered to download the WRF-Model code, distributed broadly among WRF principal partners, U. S. universities and government labs, the private sector and foreign institutions. In August, 2001, MMM organized and hosted the Second Annual WRF Users Workshop (115 participants) together with a WRF Users Tutorial (80 participants).

In the cooperative development of a complex forecast-model system, care must be taken to insure an orderly process in which the various major components are developed in a consistent fashion and integrated effectively into the overall design. Toward these ends, a Management Plan for the WRF Project was developed by the principal WRF partners and approved by the Interagency Working Group (IWG) of the U.S. Weather Research Program. This Plan established a WRF Oversight Board (Robert Gall as member) that is responsible for overall supervision of the WRF Project. The Plan also includes a WRF Science Board (Jimy Dudhia as member) that provides technical guidance to help WRF meet the needs of a broad user community, and WRF Development Teams that design and implement the various components of the overall WRF system. The WRF Coordinator (Klemp), together with the Development Team Leaders (including Joseph Klemp and Ying-Hwa Kuo), oversees the development efforts to ensure that the overall design goals are achieved and that milestones are accomplished on schedule. To provide specific focus on the major tasks within the development-team areas, approximately 15 WRF Working Groups have been created (including William Skamarock, John Michalakes, Dale Barker, Jimy Dudhia, and Tim Spangler of COMET as Group Leaders).

The WRF Oversight Board met in January to address funding commitments to WRF, and the WRF Science Board met in August, following the Users Workshop, to discuss priorities for development activities. The WRF development teams held planning workshops in February (Washington D. C.) and August (Boulder) to review the status of development efforts, and refine the plans and schedules for future work.

Further information on the WRF project, experimental real-time forecasts and the community-model release is available on the WRF web site, http://wrf-model.org/.