Overview of NCAR/MMM's 3DVAR System

1) Introduction
2) Motives For Developing 3DVAR For Use With MM5
3) 3DVAR within the MM5 Modeling System
4) 3DVAR Top-level Description
5) 3DVAR Source Code Organization
6) Useful Links

1) Introduction

This document is an overview of the NCAR/MMM 3DVAR (three-dimensional variational) data assimilation system designed initially for use with the MM5 model. More detailed descriptions of the 3DVAR algorithm, results from tests and future development plans can be found in links from the NCAR/MMM MM5 3DVAR web site.

Following this introduction, 2) Motives For Developing 3DVAR For Use With MM5 describes the justifications for this work and how it is intended to fit in with other developments e.g. incremental MM5 4DVAR, WRF 3DVAR, etc. The MM5 environment in which 3DVAR will initially be utilized is described in 3) 3DVAR within the MM5 Modeling System. In 4) 3DVAR Top-level Description an overview is given of the basic algorithms within the 3DVAR system. More detailed documentation is available from the NCAR/MMM MM5 3DVAR web site. A significant effort has been made to design the code structure to allow ease of use, code sharing with other systems, use of automatic makefile, html generation) tools, GHUI/web-based interfacing etc. This work is briefly summarized in 5) 3DVAR Source Code Organization. Finally, a list of useful links to more detailed information is given in 6) Useful Links (code, documentation, etc).

As an alternative to the following description, some may find looking at the actual code a better way to understand the system. If so, then the latest official 3DVAR code release can be downloaded from the NCAR/MMM MM5 3DVAR web site.

2) Motives For Developing 3DVAR For Use With MM5

With the pre-existence of a MM5 4DVAR research tool, it is perhaps necessary to discuss the reasons for developing a new 3DVAR system for use with the MM5. Given the short-term (1-2 year) goal of designing a variational data assimilation system for operational use at the US Air Force Weather Agency (AFWA) and for Taiwan's Civil Aeronautics Administration (CAA), the plan was to initially concentrate on producing a respectable (i.e. accurate, computationally efficient and robust) 3DVAR system for both research and operational use. The reasons for this are:

3DVAR is computationally cheaper to run that 4DVAR.
There are still many potential ways of improving VAR systems in addition to including a 4DVAR capability. These include new improved specification of background/observation errors, improved balance constraints and a host of observations that are currently under-used or ignored completely in current systems. This situation is likely to persist for the foreseeable future.
4DVAR can potentially provide an improved analysis under certain circumstances e.g. the assimilation of asynoptic precipitation observations in a relatively data-sparse mesoscale resolution domain in a baroclinically active regime. Alternatives to 4DVAR can be formulated which address some of these issues e.g. asynoptic observations can be treated in 3DVAR via a rapidly updating (e.g. hourly cycling) 3DVAR or the so-called First Guess at Analysis Time (FGAT, or 3½DVAR) method in which observation minus forecast differences are calculated using forecast states within a time-step of the observation’s validity time. Another benefit of 4DVAR, namely flow-dependent background error covariances, may also be approximated in 3DVAR through grid (e.g. isentropic, semi-geostrophic) transformations, anisotropic recursive filters and/or the use of ensemble information.
Many of the algorithms used in 4DVAR are found in the much less computationally expensive 3DVAR system (e.g. observation operators, minimization, preconditioning, multivariate, background error specification, data assimilation diagnostics). The only notable exceptions are the tangent linear (TL) and adjoint of the forecast model, required for 4DVAR. The 3DVAR system therefore provides a training ground for these crucial aspects of the data assimilation system.
All of the above techniques, applied to 3DVAR will be available to 4DVAR once that capability is built. They may also be used in applications where 4DVAR is not an option, either through lack of code or computational power.
Most 3/4DVAR systems have been initially developed for the global assimilation problem. It is prudent to consider the particular aspects of the mesoscale (e.g. differing balance, impact of moisture, convection, etc) before following the path of other systems' development.
It should be relatively easy in future to merge the existing 3DVAR with the TL and adjoint code to produce an incremental 4DVAR system. This system would permit the potential benefits of 4DVAR to be tested whilst keeping the computational cost to an affordable level (incremental 4DVAR is operational at ECMWF and Meteo-France at a cost of between 5-10 x 3DVAR). The choice between running 3DVAR or 4DVAR mode can be made at compile or even run-time.

In designing a 3DVAR data assimilation system for MM5, the goals are:

Improved MM5 forecasts relative to those integrated from analyses produced from current assimilation schemes used e.g. LITTLE_R, RAWINS.
Computationally efficient and robust system.
Operational implementation at Taiwan's CAA and AFWA by 2002.
Plug-compatibility of code to extensions e.g. new observations, improved background error covariances, incremental 4DVAR.
Extensive use of FORTRAN 90 and WRF coding standards.
Flexible to computational environment: differing platforms, processor design (e.g. vector/scalar), shared/distributed memory.
Applicability of code to the WRF 3DVAR system. Development plans for the WRF 3DVAR precede plans for 3DVAR with MM5 (start January 2000). It was therefore acknowledged at the start that any code developed in an application of 3DVAR for MM5 should be useable as/within a prototype WRF 3DVAR system, otherwise there would be little point in developing a system that would be replaced within a year or two. The 3DVAR application for MM5 has adopted WRF capabilities as soon as they became known e.g. specified observation operators, extensive use of FORTRAN 90, flexibility to vertical co-ordinate/control variables, grid and observation space formulism (this latter flexibility was later deemed low priority by the WRF 3DVAR group and removed). These efforts were felt crucial, given the negligible resources available for WRF 3DVAR at other centers pre-2002.

3) 3DVAR within the MM5 Modelling System

Development of a 3DVAR system for use with MM5 began in earnest in January 2000. New versions of 3DVAR are produced every few months following extensive “regression tests”. The 3DVAR system sits within the MM5 environment as shown in Figure 1.

The three 3DVAR input data files are:

The first guess (or background) field x^b: This is a previous forecast used as a starting point for the computation of the current analysis. At present, the system uses the pre-existing MM5 preprocessing packages TERRAIN (defines domain, orography, land use etc), PREGRID (reads background forecast from various sources e.g. MM5, RUC, ETA, AVN etc), REGRIDDER (horizontally interpolates background to 3DVAR grid) and INTERP (vertically interpolates background to 3DVAR grid). In “cycling” or warm-start” mode, the first guess is a previously run MM5 forecast from the current domain.
The background error statistics B: This file contains the error covariances associated with a given input background field. Currently these are climatological values estimated using the `NMC'-method of averaged forecast differences. These may be tuned by comparison with O-B data. A potential source of improvement is to use ensemble information, rather than the NMC-method, to specify B.
The observations y^o: Observation data is read into a 3DVAR observation preprocessor (3DVAR_OBSPROC) in “LITTLE_R” format (the same observation format previously used by the MM5 LITTLE_R package). Gross quality control (QC) is performed (domain/time/duplicate checks, station blacklist etc). The possibility for multiple, nested domains are catered for in the observation preprocessor by serial applications of the observation preprocessor to smaller and smaller domains. This is deemed important to reduce I/O with the large observation numbers foreseen in a multiple scale, radiance-heavy, multi-domain application of 3DVAR. Selected data is output from the observation preprocessor in “3DVAR” format (BUFR will be used in the WRF era) ready for 3DVAR.

Figure 1: The Use Of 3DVAR within the MM5 model environment:

Given these three source of information, 3DVAR performs a cost-function minimization (described elsewhere) in order to produce a set of "optimal" analysis increments x' which, when added to the background field x^b and optionally initialized, produce the analysis x^b x^a =x^b + Ix' used as initial conditions in subsequent forecasts (I is an optional initialization operator). In addition to outputting the analysis (and optionally the analysis increments too), a number of diagnostic output files are produced containing e.g. results of adjoint/inverse tests, background error information, fit of background/analysis to observations, cost function and gradient evolution, CPU/memory breakdown, etc.

The 3DVAR analysis may include increments at all points, including the lateral boundaries. There is therefore a mismatch between lateral boundary conditions (the BDYOUT file) and the initial conditions. This is corrected using the 3DVAR “UPDATE_BC” utility, which modifies the tendencies at the initial in the BDYOUT file to match the analysis fields. In future, the 3DVAR output may also be post-processed top perform the following additional tasks:

Strip out extra rows/columns/levels if 3DVAR has been run over a larger domain than the subsequent forecast (used in cold-start runs).
Interpolate increments to forecast grid (used if 3DVAR is run at lower resolution).
Perform external initialization via e.g. a digital filter or an incremental analysis update (IAU).

4) 3DVAR Top-level Description

This section gives an overview of the 3DVAR system itself. As an alternative to the following description, some may find looking at the actual code a better way to understand the system. Is so then visit the NCAR/MMM MM5 3DVAR web site. In addition, the code itself contains inline comments intended to help the use understand the process. Figure 2 illustrates the various stages of the 3DVAR algorithm. The function of each is stage is summarized as follows:

a) Setup MPP: Uses WRF framework’s distributed memory capability to initialize tile, memory, patch dimensions, etc.

b) Read Namelist: Reads namelist file to set 3DVAR options. The namelist file is created automatically on running the 3DVAR shell script. This capability is intended to allow easy interface of 3DVAR with GHUIs and/or java/web-based control of 3DVAR in future versions.

c) Setup First-Guess: Reads in the first-guess (also called the background) field, extracts necessary fields and creates the background FORTRAN 90 derived data type “xb” (e.g. xb % idim, xb % u(:,:,:) etc.).

d) Setup Background Errors: Reads in background error statistics (currently MM5 format), extracts necessary quantities (eigenvectors, eigenvalues, length scales and regression coefficients for each control variable) and creates the background error FORTRAN 90 derived data type “be” (e.g be % psi % variance, be % psi % eigenvalue, etc.

e) Setup Observations: Reads in observations (ASCII “LITTLE_R” format for MM5, BUFR for WRF), and creates observation FORTRAN 90 derived data type “be” (e.g. ob % synop, ob % sound % u(:), ob % ssmi etc.

f) Calculate Innovation Vector: Calculates model equivalent B of the observation O, and compute observation minus background (O-B) difference. Creates innovation vector structure “iv”.

g) Minimise Cost Function: Produces "optimal" control variable values v. This is the core of the 3DVAR algorithm and described in more detail in the MM5 3DVAR Technical Note.

h) Compute Analysis: Converts minimized control variables into model space analysis increments.

i) Calculate Diagnostics: Compute O-B, O-A statistics for all observation types and variables, and A-B statistics for all model variables and levels.

j) Output Analysis: Outputs analysis in native model format. Also optinally outputs analysis increments for diagnostic purposes.

k) Tidy Up: Deallocate dynamically-allocated arrays, etc. Output diagnostics: Numerous files produced containing a wealth of information on how 3DVAR has performed.

Figure 2:Top-level overview of the 3DVAR process.

The “outer-loop” permits the recalculation of the innovation vector (O-B) and reanalysis using the latest “analysis guess” as the new background. This can be used to allow for nonlinearities (e.g. in observation operators, balance equations, or (in 4DVAR) the forecast model).

5) 3DVAR Source Code Organization

One of the benefits of using FORTRAN 90 is that it allows use of longer subroutine and variable names. An effort has therefore been made to use meaningful (sometimes longer) names to allow easy reading of the code.

The 3DVAR source code is split into subdirectories containing logically distinct algorithms. Figure 3 is an example - the 3dvar/da_3dvar/src subdirectory (this level corresponds to the original top-level 3DVAR directory in the original serial code before 3DVAR was made a core for WRF). As well as making the 3DVAR code easier to follow, the idea is to identify aspects that could be used, replaced or shared with code outside the 3DVAR system e.g. general meteorological, interpolation, minimization routines.

Figure 3: 3DVAR source code organization for subdirectory 3dvar/da_3dvar/src.

Each subdirectory within Figure 3 is identified with a particular FORTRAN90 module i.e. all the routines within e.g. the DA_Sound subdirectory are "INCLUDEd" in a single module file with the same name (and file-type .F) in that subdirectory. Figure 4 gives an example for the subdirectory DA_Sound.

Other reasons for adopting this code structure include the use of available automatic makefile generation scripts (which search .F files and .inc routines specified in their INCLUDE lines). Also, experience has shown that this approach makes use of automatic fortran->html tools much easier - common subdirectory, file and subroutine naming conventions are required to utilize this very useful facility.

Figure 4: 3DVAR subdirectory 3DVAR/da_3dvar/src/DA_Sound.

The DA_Sound.F file is shown in Figure 5.

Figure 5: Example 3DVAR module – DA_Sound.F.

6) Useful Links

MM5 3DVAR web page - http://www.mmm.ucar.edu/mm53dvar/
MM5 web page - http://www.mmm.ucar.edu/mm5
WRF 3DVAR web page - http://wrf-model.org/WG4

Please contact mesouser@ucar.edu if you have any questions concerning NCAR/MMM's 3DVAR system.

Last Modified: June 2003