-------------------------------------------------------------
Prototype WRF (Weather Research and Forecast) model. 8 March 1999
First try, constructed by
W. Skamarock (skamaroc@ncar.ucar.edu, (303) 497-8893)
J. Michalakes (michalak@ncar.ucar.edu, (303) 497-8199)
J. Dudhia
D. Gill
J. Klemp
--------------------------------
-> Outline
This document is organized into the following sections:
-> Outline
-> Preface -
-> WRF Model Features needing consideration -
-> Compiling and running:
-> WRF model specifics -
--> Nomenclature:
--> Software architecture:
--> WRF call tree:
--> Data Registry:
---> Variables that are designated Part 3d, 2d, or 1d:
---> Variables that are designated Part 3d_chem or 3d_moist:
---> Variables that are designated Part '-':
---> Variables that have non '-' entries the Decoup field:
---> Variables that have a non '-' entry for the Tendency field.
--> Namelist organization:
--> Parallelism
-> Notes:
-> Preface -
In this directory structure sits the first prototype WRF model, written
by Skamarock, Michalakes, Dudhia, and Gill using a solution technique
developed by Klemp, Skamarock and Dudhia.
The WRF modeling systems, outlined in the WRF model working group
reports, is comprised of preprocessors for data assimilation, model
initialization etc., a model that integrates the equations of motion on
some grid or system of grids, and post-processors for examining the
model forecasts. This prototype model is a prototype of the
integration engine (which we call the "model"). It is not a
full-fledged model. Presently, it capabilities and limitations are:
1) The solver within the model integrates the dry fully compressible
equations of motion.
2) Shared memory and distributed memory parallel processing
capabilities, making use of domain-decomposition methods,
have been implemented in this prototype.
3) The model has the capability of integrating multiple nested
grids, but the routines which will link the grids
(essentially interpolation routines that pass data of various
forms between the grids) have not yet been constructed.
The data structure that stores the grid fields and the algorithms
that control the integration sequence exist and are functioning
in this model version.
4) The model contains no physics, and uses periodic lateral boundary
conditions and rigid free-slip surfaces at the bottom boundary
and upper rigid lid.
5) This model uses low-order numerics (2nd order centered time
and space differencing) and a split-explicit time-integration
scheme.
6) The model does carry around (and advect/integrate) an arbitrary
number of scalars representing moisture and an arbitrary
number of scalars representing chemical species, the former for
use in a moist model and the latter for use in air quality and
atmospheric chemistry studies.
7) With the exception of namelist input and some rudimentary output,
the model is essentially I/O-less. Initial conditions are
idealized cases that are computed in the model itself; output
consists of field dumps, either ascii dumps or gmeta (NCAR
graphics) for the purposes of initial development and debugging.
This prototype is not meant to be used for forecasting or science at
this stage. The prototype is being released for the purpose of allowing
all interested parties to examine the programming constructs and coding
style we have employed. We welcome feedback from anyone who wishes to
comment on the prototype. We would like to critically evaluate the
prototype at this early stage in model construction, with the goal of
incorporating improvements and removing problematic constructs, before
significant code is developed for more sophisticated models.
The data structure and the algorithms for managing multiple grids can be
considered to be fairly complete in most aspects (outside of the
routines that communicate data between the grids). While many parts of
the discretization will definitely not be those employed in the WRF
model, the time-integration scheme represents a class of schemes which
may become the basis of the WRF model. Hence the solver in the
prototype, in particular the top section where the time-integration of a
grid is controlled, should also be considered fairly complete and
representative of what might actually be used.
A major part of the new WRF code, documentation (both in-code and
offline) is not complete. We would welcome comments on where more
in-line documentation is needed, and you are presently looking at almost
all of the offline documentation that exists. Please understand that we
did not want to spend lots of time documenting code that would likely
be discarded later, and we wanted to get this initial prototype design
out to everyone as soon as we could.
--------------------------------
-> WRF Model Features needing consideration -
As noted above, this isn't meant to be the WRF model with respects to
the numerical discretization we've employed. There are many aspects of
the model architecture that we are very interested in critiquing.
1) The use of Fortran 90/95.
2) The use of cpp for some model configuration choices.
3) The use of PERL for automatic generation of critical parts
of the data structure using user-specified variable lists.
4) The parallel processing implementation for shared and/or
distributed memory parallelism.
We welcome comments on any other model features as well. In particular,
we need to consider/decide on the index ordering within the model, and
I/O methods/formalism and file structures for the model gridded data.
---------------------------------
-> Compiling and running:
1.
Edit the configure.wrf file. Move the settings specific to your
target architecture the bottom of the first section. This file
is included by src/Makefile so the syntax and semantics are
those of the Unix make utility.
Name of the Fortran90 command:
FC = f90
Fortran90 compiler flags:
FCFLAGS = -inline all -automatic -cpp -fast -free -I. -O4 -arch host -omp
List extra .o files that may be required for this architecture:
EXTRAMODULES =
Flags seen by CPP and by the perl script "use_registry":
ARCHFLAGS = -D_OPENMP
Perl command:
PERL = /usr/local/bin/perl
Name of the registry file in the src directory:
REGISTRY = Registry
Libraries (e.g. NCAR graphics):
LIB = -L/usr/local/ncarg/lib -lncarg -lncarg_gks -lncarg_c -lX11 -lm
Link-editor flags:
LDFLAGS = -fast -automatic -convert big_endian -O4 -omp
CPP command:
CPP = /usr/bin/cpp
Flags to CPP:
CPPFLAGS = -I$(LIBINCLUDE) -C -P $(ARCHFLAGS) # -DNONCARG
2. cd to the src directory
3. type 'make'
4. The resulting executable is named wrf.exe, which on workstations
may be run directly. On parallel machines, the command for
starting a parallel job may differ.
The 'make clean' command will delete intermediate, object files,
executable files, automatically generated files, etc.
---------------------------------
-> WRF model specifics -
--> Nomenclature:
Solver Code called to integrate a domain forward through one time step
Patch A subset of a model domain allocated to a single address space.
This is the entire domain unless distributed-memory parallelism
is employed.
Tile A subset of a patch executed by one thread. This domain
subset is bounded spatially and temporally by the amount of
computation that may occur without concern for coherency
before horizontal dependencies arise. No assumptions may
be made inside the tile code about the order in which tiles
will execute.
Process An operating system construct: one address space,
containing one or more threads of execution. A patch is
computed by a process.
Thread A series of instructions through a code (plus a small
amount of local data -- stack and thread-local heap
storage; only the former is used in WRF). Multiple threads
can exists through a single block of code -- say, a
parallel DO loop, and they can all access shared data
within the process. In WRF, a thread is what executes a
tile. There may be more threads than tiles or more tiles
than threads. In the former situation, some threads will
be idle; in the latter, each thread will execute several
tiles in sequence.
State Data that persists over the life of a domain (across
multiple calls to the solver). May be communicated between
patches and shared between threads.
I1 Data that persists over the life of a single call to the
solver but which need not persist across calls to the
solver. Tendency data would be an example. May be
communicated between patches and shared between threads.
I2 Data that persists only for the life of a single call to a
package within the solver. Strictly "local" data. May not
be communicated nor shared between threads.
Logical domain dimensions
This is the logical size of the model domain, exclusive of
extra points for any physical boundary region (e.g.
periodic). In a model subroutine, these come in through
the arguments: ids, ide (west/east start and end of
domain); jds, jde (south/north start and end of domain);
and kds, kde (vertical start and end of domain). In the
case of staggering, the logical domain dimensions are large
enough to hold the largest dimension of any field in that
dimension. In the subroutine, these may be used for
boundary tests or other instances where the position of an
index relative to the global domain is needed. These
should not be used in loop ranges except within max/min
intrinsics for boundary testing, nor should they be used to
dimension arguments or memory in the subroutine. Starts
and ends of dimensions are expressed in global
coordinates. In the current implementation, i runs west to
east with ascending indices; j runs south to north with
ascending indices; and k runs from bottom to top with
ascending indices.
Patch domain dimensions
These are the dimensions of the logical patch -- the set of
points that are assigned to be computed locally on this
process, not including halo or boundary regions. Patch
dimensions are thus distinct from memory dimensions (below)
in that memory dimensions must be greater than or equal to
patch dimensions. Patch dimensions are available at the
solver level but are generally not passed to model
subroutines in this version of the prototype. The
dimensions ips, ipe; jps, jpe; and kps, kpe and may be
passed into routines at the author's discretion. They may
be used for testing for patch boundaries; they should not
be used as loop ranges. Starts and ends of dimensions are
expressed in global coordinates.
Memory domain dimensions
This is the memory size of the model domain inclusive of
all additional memory in each dimension (for halos,
boundaries, paddings for cache alignment, etc.). In a
model subroutine, these come in through the arguments: ims,
ime (west/east start and end of domain); jms, jme
(south/north start and end of domain); and kms, kme
(vertical start and end of domain). Starts and ends of
dimensions are expressed in global coordinates. In the
subroutine, these are used for dimensioning argument
arrays, local arrays, etc. Use of these for any other
purpose is strongly discouraged, since the subroutine may
make no assumptions about the actual size of the memory
other than that it is sufficiently large.
Tile domain dimensions
This is the computational extent of the model domain within
a model subroutine. It includes only the cells that are to
be computed in that call to the subroutine. Though this is
to be avoided, the subroutine may, at the author's
discretion, chose to compute outside this range, but the
dimensions passed into the routine are those for which the
routine is responsible for computing. These come in to the
subroutine though the arguments its, ite (west/east start
and end of domain); jts, jte (south/north start and end of
domain); and kts, kte (vertical start and end of domain).
Starts and ends of dimensions are expressed in global
coordinates.
Halo
Extra memory allocated solely for the purpose of
interprocessor edge data exchanges between patches if
multiple patches exist. The memory domain dimensions
subroutine must be larger than the patch dimensions by at
least the amount necessary to allow for halos on one or
both sides of the patch in that dimension.
Physical Boundary Region
Points that extend the logical domain boundaries by a
sufficient amount to permit certain boundary conditions:
symmetry, periodic. In the case of patches that have at
least one side on the edge of a logical domain, the memory
domain dimensions must be larger than the patch dimensions
by at least the amount necessary to allow for a physical
boundary region.
--> Software architecture:
The model calling structure is organized both horizontally and
vertically. As described elsewhere [1], the horizontal organization is
into layers: the driver layer, the model layer, and a mediation layer.
The horizontal stratification of the model architecture provides
separation of concerns between the driver layer -- encapsulating
hardware architecture, shared and distributed memory parallel , input
(namelist, initial data, etc.) and output concerns -- and the model
layer, which consists of strictly computational routines of the code,
written to be callable for a single "tile". Between the driver and the
model layers sits the mediation layer, the "glue" layer which knows
something about both the other layers. Obviously we try to keep this
layer as thin as possible.
The vertical organization of the call tree is into sections by broad
function: basic initialization, configuration, initialization, time
integration, and model output. In schematic, the overall organization
of WRF is as follows:
basic
init. config. init. integ. i/o
+=======+==========+=========+========+========+
| | | | | |
Driver | | | | | |
| | | | | |
| | | | | |
+=======+==========+=========+========+========+
Mediation | | | | | |
+=======+==========+=========+========+========+
| | | | | |
| | | | | |
Model | | | | | |
| . | . | . | . | . |
. . . . .
. . . . .
What this diagram conveys is that each section of the model will enlist
driver layer, model layer, and mediation layer components to perform
its function. (A fair amount of effort has been invested in developing
this WRF prototype code in *actually adhering* to this conceptual
organization, though there are sure to be some occasional divergences
in this early initial version.) One key motivation for this
organization is reuse: the model layer may be interfaced to other
architecture- or application-specific driver layers, including the use
of computational frameworks. Likewise, the driver framework should be
reusable with other model layers. Limiting information about
parallelism, for example, to the driver and mediation layers, allows
the investment in model fortran routines to be preserved and exploited
with efficiency on a range of diverse computer architectures.
Although some effort has been made to provide flexibility in the choice
of array index order in the driver layer, this has been essentially
fixed to k-innermost, i, then j-outermost in the model layer. The
reason for this is that there are no constructs or mechanisms in the
Fortran language itself that allow for reordering these without
recoding. A suitably engineered source translation engine could handle
this easily as a pre-processing step prior to compilation.
--> WRF call tree:
wrf() [wrf.F ; Driver ]
|
+-- init_modules() [ init_modules.F ; Driver/basic-init ]
|
| Each module that is "used" by wrf provides a
| routine in its CONTAINS section called init_.
| This routine calls all of these. If one adds a new module to
| WRF, such a routine should be provided in the module (may be a
| stub) and a call statement inserted in this routine.
|
+-- initial_config() [ module_configure.F ; Driver/config ]
| |
| +-- read_namelist_data() [ module_configure.F ; Mediation/config ]
|
| Takes as input a fortran unit number and an empty instance of
| a model_config_rec_type and returns the structure populated with
| model configuration information.
|
| [ Registry interfaces (see Sec 4 in src/use_registry): ]
| [ ]
| [ #include ]
| [ #include ]
| [ #include ]
| [ #include ]
|
+-- alloc_and_configure_domain [ module_domain.F ; Driver/config ]
| |
| | Given a domain id, a model_config_rec_type structure, a pointer to
| | a grid and parent domain type (if applicable), configure and allocate
| | space for this new domain and hook it into the nest hierarchy.
| |
| | [ Registry interfaces (see Sec 4 in src/use_registry): ]
| | [ ]
| | [ # include ]
| | [ # include ]
| | [ # include ]
| | [ # include ]
| |
| +-- patch_domain [ module_domain.F or elsewhere ; Driver/config ]
| |
| | Given a domain id, a parent id (if avail), the logical domain
| | dimensions, and the width of the physical boundary region around
| | the logical domain, calculate a decomposition and return the
| | memory domain dimensions and patch domain dimensions necessary
| | on each processor.
| |
| | The version of this routine in module_domain.F is for the
| | trivial case where there is only one patch and so, there is
| | effectively no decomposition. It merely sets the patch dimensions
| | the same size as the logical domain dimensions and then adjusts
| | the memory dimensions to be large enough to contain any physical
| | boundaries that may have been specified via the namelist.
| |
| | As distributed memory capabilities are added to WRF, they go
| | into source files named module_dm.F and this module should supply
| | a routine to replace the patch_domain routine defined here. For an
| | example, see the existing module_dm.F.
| |
| +-- alloc_space_field() [ module_domain.F ; Model/config ]
| |
| | Given a pointer to a new domain data structure of TYPE(domain)
| | and the memory domain dimensions, allocate 2-, 3-, and 4-dimensional
| | data structures. When grid is returned, it's fully allocated.
| |
| +-- define_comms() [ module_dm.F ; Driver/config ]
|
| DM_PARALLEL only. This is a call to a routine in the module_dm.F
| library that can be used to set up communication on domain state
| data fields, now that they are allocated.
|
+-- init_domain [ module_initialize.F ; Mediation/init ]
| |
| | Given a grid data structure, initialize the fields therein. This
| | will eventually involve I/O but the WRF prototype initializes from
| | idealized cases that are computed here and in subroutines.
| |
| | DM_PARALLEL only. Calls to routines in module_dm.F may occur
| | here to update halos and periodic boundary conditions when
| | domains are decomposed over multiple patches. The routines
| | called now are wrf_dm_halo() and wrf_dm_boundary(), defined in
| | module_dm.F. Here and other places requiring communication,
| | comments are included showing the fields that need to be
| | communicated and the stencils on which data need to be
| | exchanged. These are informational only.
| |
| +-- init_coords() [ module_initialize.F ; Model/init ]
| +-- init_base_state() [ module_initialize.F ; Model/init ]
| +-- init_state() [ module_initialize.F ; Model/init ]
| +-- set_physical_bc() [ module_bc.F ; Model/init,integrate ]
|
| This is a routine that updates boundary conditions (periodic,
| symmetric) if the boundaries can be updated without
| communication. Otherwise the wrf_dm_boundary() routine is
| assumed to have updated the boundary region.
|
+-- integrate [ module_integrate.F ; Driver/integrate ] << RECURSIVE >>
|
| This routine takes a grid, a starting and an ending time value and
| updates grid and all of its children from the starting time to the
| ending time. It handles nests (as children of grid) and also
| overlapping domains on the same nest level (as siblings of grid).
|
+-- solve_interface [ solve_interface.F ; Mediation/integrate ]
. |
. | This call takes a single argument, grid, which contains all
. | the state data for a single domain, and when
. | solve_interface returns, the grid has been updated one time
. | step. From this point down in the call tree, the scope of
. | the computation is a single domain. Solve-interface's main
. | purpose is to be the dereferencing agent for grid, so that
. | above this routine in the call tree, the domain is known
. | only as a single name. Below this routine, in solve, all
. | data associated with the grid is passed through argument
. | lists. The reason for this is partly for performance to
. | minimize the cost of dereferencing the grid data structure
. | to once per time step. Secondarily, this allows the solver
. | to be written to pass data exclusively through argument
. | lists, so that other drivers can be interchanged, whether
. | or not the new driver represents the domain as a single F90
. | structure (computational frameworks, for example, may
. | instead represent the grid as multiple distributed
. | arrays). At the level below solve interface, it is not a
. | concern how the driver is storing the state data for a
. | domain.
. |
. | The dereferencing and the tedious and error prone listing
. | of the arguments and their fields in the grid data structure
. | is automated using the Registry mechanism. Thus, if the
. | set of state fields in the model changes, it is not necessary
. | to reconstruct the dereferencing or actual argument lists here
. | or below in solve, where the dummy arguments are also listed
. | using a Registry-generated file.
. |
. | [ Registry interfaces (see Sec 2 in src/use_registry): ]
. | [ ]
. | [ #include "solve_actual_args.inc" ]
. |
. | This routine also increments the time counter for the
. | grid.
. |
. | This routine includes an interface block definition of the
. | solver that it calls, called solve.int.
. |
. +-- solve [ solve.F ; Mediation/integrate ]
.
. The top level flow of control over a single time step is
. here.
.
. This routine is the principle routine of the mediation
. layer, containing some information that is of concern to
. the the driver layer and some information that is of
. concern to the model. Everything below this layer of the
. code is considered Model.
.
. There are no communication or threading calls or directives
. below the solve subroutine.
.
. All state data is passed to solve through its argument list.
.
. [ Registry interfaces (see Sec 2 in src/use_registry): ]
. [ ]
. [ #include "solve_dummy_args.inc" ]
. [ #include "solve_dummy_arg_defines.inc" ]
.
. I1 data (tendencies, decoupled versions of prognostic
. variables) are defined in solve and allocated on the
. program stack. This avoids the need to have more than a
. single version of these occupying memory at a time, even
. when multiple domains are being simulated.
.
. Virtually without exception, no computation is done in
. solve, rather it determine the tiling of the domain with a
. call to get_tiles(), and then every section of the
. underlying model is called within loops over tiles and
. these loops may be multi-threaded using DO PARALLEL
. directives. Put another way, anything that is called from
. solve is called for a tile (the tile may be the entire
. patch, or it may be subdivided into smaller units that can
. be assigned to multiple threads). The routines below solve
. are written to be callable for a single tile and by
. definition do not involve horizontal data dependency.
. Whenever the computation reaches a point where horizontal
. data dependency becomes an issue, control reverts to the
. the solve level -- any communication that needs to occur is
. performed using calls to wrf_dm_halo and/or wrf_dm_boundary
. and then the next possibly multi-threaded loop over tiles
. is initiated.
.
. The standard interface (formatting and style is not
. addressed here) to a Model subroutine called from this
. layer and below is:
.
. SUBROUTINE sub ( arg1, arg2, ... argn, &
. ids , ide , jds , jde , kds , kde , & ! domain dims
. ims , ime , jms , jme , kms , kme , & ! memory dims
. optional-> [ ips , ipe , jps , jpe , kps , kpe , & ! patch dims ]
. its , ite , jts , jte , kts , kte ) ! tile dims
. IMPLICIT NONE
. ! 3D State arrays
. REAL , DIMENSION ( kms:kme,ims:ime,jms:jme ) :: arg, arg, ..
. ! 2D State arrays
. REAL , DIMENSION ( ims:ime,jms:jme ) :: arg, arg, ..
. ...
. DO j = jts , jte ! Loop ranges use tile dimensions
. DO i = its , ite
. DO k = kts , kte
. ...
.
. Currently in the WRF prototype, the TYPE(domain) "grid"
. argument is also passed, but this may be eliminated.
.
R-- Integrate() calls itself for subdomains (nests) and other
domains on same nest level.
--> Data Registry:
The Data Registry is an additional component of the WRF software
architecture, which perhaps does not fall easily into the
Driver/Mediation/Model taxonomy, but which provides a mechanism for
abstracting information about model data in table form, and then
propagating this information across numerous modules and interfaces
between modules in the code so that, for example, adding or deleting a
state variable or modifying the namelist semantics is a one-line change
to the Registry table and a recompile. Currently, the WRF Registry is
implemented using an ASCII table named "Registry", a PERL script, named
"use_registry", and the CPP preprocessor. The use_registry script
in the src directory contains detailed documentation of what files
are generated from the Registry table, and where these are used in
the code. What follows here is a description of the form of a Registry
table entry and what this means for the code at-large.
A registry entry is a line in the registry table and each field
(currently there are ten) are separated by white space (one or more
spaces or tabs). A sample registry entry, which adds the state
variable for the u-component of winds, is:
state real ru 3d 2 X u ru_tend init_domain
Note that the tenth field, a default value used for namelist entries,
is missing for ru, a prognostic variable. The fields if a state entry
are:
Field Allowed values Other information
1. Code state This flags the registry entry
as an element of the state data
table. currently the state data
table is the only table represented
in the Registry file.
2. Type real, integer, ... The type of the state data variable
3. Sym The name of the variable (ru in
the example)
4. Part 3d What part of the TYPE(domain) this
3d_moist variable occupies.
3d_chem
2d
1d
-
5. NumTLev 1, 2, 3, or - Number of time levels for variable.
6. Stagger X, Y, Z, or - Staggering dimension or - if not
staggered.
7. Decoup The name(s) to use for decoupled
or - forms of the variable, if applicable.
8. Tend The name to use for the tendency for
or 'yes' this variable, or, in the case of
or - moisture and chemistry fields, whether
to have a tendency array.
9. Whereinit Where the variable is first initialized.
or In the case of namelist variables,
or namelist_derived the namelist record identifier is specified.
In the case of configuration variables that
whose value is derived from the namelist
but which do not appear in the namelist
themselves, the string namelist_derived
is specified.
10. Default initial value applicable to namelist variables that
actually appear in the namelist only.
The following is additional discussion of the Registry entry
fields listed above.
---> Variables that are designated Part 3d, 2d, or 1d:
If the Registry "part" name for a variable is 3d, 2d, or 1d, then it is
stored in a field named "data_" in the TYPE(domain). The
registry mechanism also generates a defined integer constant whose name
is:
ENUM__[_].
These constants may be referenced in any subroutine that USEes
module_state_description. For example, a reference to the
first time level of variable RU is:
grid%data_3d[ENUM_3D_RU_1]%data
For part 3d, 2d, and 1d, variables that have only 1 time level,
the reference will be (using the 3D field 'PIB' for example):
grid%data_3d[ENUM_3D_PIB]%data
It is rarely necessary for the programmer to be concerned with these
structures, since they are employed only above the Model layer of the
code. At the interface between the Driver layer and the Model layer,
these references are automatically generated and they appear only in
the routine solve_interface(). In the solve() routine and below, the
variables are passed explicitly through argument lists and are known by
their Registry symbol name, with the time level appended in the case
of multiple time level variables.
RU, which has 2 time levels, is known simply as RU_1 and RU_2 in
solve() and below. Variables that have only 1 time level, such as
PIB, are known simply by their Registry symbol name in solve and below.
---> Variables that are designated Part 3d_chem or 3d_moist:
Moisture and chemistry fields are treated differently because there are
many operations on these that can be performed en masse. They appear
in the model as four-D arrays, and the fourth dimension is "species".
There are separate four-D arrays for each time level.
If the Registry "part" name is 3d_moist, the field is stored as species
in the field data_3d_moisture_ where timelevel is 1, 2, or 3
(See note (b) for discussion of time-levels). Likewise, if the
Registry part name is 3d_chem, the array is a species in the field
data_3d_chemistry_. The index into the four-d moisture or
chemistry field array is given a defined constant with the name:
ENUM__
Again, these constants may be referenced in any subroutine that USEes
module_state_description. Since there are separate four-D arrays for
each time level, the time level is not encoded into the name of the
defined constant. Thus, a reference to the first time level of
moisture field Q0 is:
grid%data_3d_moisture_1(:,:,:,ENUM_3D_MOIST_Q0)
above solve() and just:
moist(k,i,j,ENUM_3D_MOIST_Q0)
within solve() and below.
---> Variables that are designated Part '-':
If the Registry "part" name is '-', then the variable is stored
as itself in the field. For example, the zero-dimensional variable
named ZETATOP is known as grid%zetatop in solve_interface() and above
and simply as ZETATOP in solve() and below.
---> Variables that have non '-' entries the Decoup field:
RU in WRF is Rho-U. But it is also used in the solver as just U,
decoupled from Rho (prognosed density). Rather than decouple the
variable each time this form is needed, I1 storage is declared in Solve
for just U. This is done automatically through the Registry. The
'Decoup' field of the Registry entry for RU specifies the name of the
decoupled form, which causes solve() to include a local declaration of
the three-D variables U_1 and U_2, for each time level. If only one
time-level is specified, then the the _1 suffix is omitted from the
variable name. Multiple decoupled variables are permitted for each
Registry symbol. For example, in the current src/Registry table,
the variable RTP has three decoupled forms: TP, T, and RT. Declarations
are automatically generated for TP_1, TP_2, T_1, T_2, RT_1, and RT_2.
Currently, one cannot specify decoupled I1 variables
for part 3d_moist and part 3d_chem variables.
---> Variables that have a non '-' entry for the Tendency field.
Tendency arrays have the same dimensionality as the variable for which
they appear. There is only ever one time level. If a tendency
variable is specified for a part 3d, part 2d, or part 1d variable,
a local declaration of the tendency array is generated in solve().
For example, the Registry entry for RU specifies a tendency array
named RU_TEND. A declaration for this is automatically generated
and RU_TEND appears as a local array in solve().
In the case of part 3d_moist and 3d_chem variables, the tendency array
is also four dimensional. Thus, if the Registry entry for any of these
specifies a tendency array, they all get one. These four-D tendency
arrays appear as local arrays in solve() and they have the names
moist_tends and chem_tends, respectively. The index into the tendency
array used for a particular species is the same defined constant
as is used for accessing the variable. For example, the tendency
for moisture variable Q0 is:
moist_tends(:,:,:,ENUM_3D_MOIST_Q0)
--> Namelist organization:
There are four records in the namelist: namelist_01, namelist_02,
namelist_03, and namelist_04. The first record, namelist_01, is for
model-wide settings and the elements of namelist_01 are
zero-dimensional (e.g. time_step_max, the number of coarse domain time
steps to execute for the simulation). The elements of the remaining
namelist records contain per-domain settings. Each element of these
remaining records is a 1-dimensional array of size 'max_dom' (max_dom
is a model-wide setting read in as part of namelist_01). Each element
of the 1-D namelist_02, namelist_03, and namelist_03 records is a
setting for a given domain.
Namelist elements are considered state and are listed in the Registry
file. The "Whereinit" field for a namelist element is either
namelist_01 (in which case it is considered a zero-dimensional,
model-wide setting) or namelist_02, namelist_03, or namelist_04, in
which case the namelist element is a 1-dimensional vector of max_dom
elements. Configuration data items that are not in the namelist itself
but that are derived from namelist variables (and passed around in the
model within the model_config_rec_type configuration record) should
also be listed in the Registry file, with a "Whereinit" field of
namelist_derived. These namelist_derived fields will be included in
the model_config_rec_type'd configuration record but won't be expected
(or allowed) in the namelist input file itself. A reasonable place to
generate derived namelist variables from input namelist variables is
within or below the routine read_namelist_data.
--> Parallelism
Mechanisms for distributed-memory and shared-memory parallelism
are isolated to a single routine of the Mediation layer, solve().
Distributed memory parallelism is encapsulated within a module,
module_dm.F . We envision multiple instances of module_dm.F, and
the choice of which to use will be made by the make routine.
The rest of the code calls routines provided in module_dm.F for
such services as halo-exchanges, periodic boundary communication,
basic decomposition. There are presently no real I/O routines
in WRF, however once these functions appear, they will also be
provided by the module_dm.F for a particular architecture and/or
message passing software layer. All calls to such routines in
and USE declarations for this module are guarded by CPP conditional
compilation directives:
#ifdef DM_PARALLEL
#endif
Solve also contains OpenMP parallelization directives around
tile-loops.
-> Notes:
[1]
``Design of a next-generation weather research and forecast
model,'' with J.Dudhia, D. Gill, J. Klemp, and W. Skamarock, in
proceedings of the Eighth Workshop on the Use of Parallel
Processors in Meteorology, European Center for Medium Range
Weather Forecasting, Reading, U.K., November 16-20, 1998. This is
ANL/MCS preprint number ANL/MCS-P735-1198. Preprint:
> ftp://info.mcs.anl.gov/pub/tech_reports/reports/P735.pdf
Slides: