Introduction

The APL Convection Mapping software consists of a number of programs that are used to process SuperDARN data to produce Global Ionospheric Convection Maps. Rather than perform the analysis as a single stage, monolithic task, the process is program down into simple stages that are calculated by individual programs. This has the advantage that the user can modify the analysis at any stage without having to re-process the data in its entirety. At first, you may find that the number of steps involved in the analysis is confusing and cumbersome, however once you have become familiar with the software you will be able to string multiple stages together feeding the output of one step into the input of another.

Generating Grid Files

Before you can start generating convection maps you must generate a grid file. These contain filtered line of sight velocity vectors that have been fitted to an equi-area global grid. A file usually contains data from one or more radar and multiple grid files are combined to generate the input required for the convection map analysis. If you have a handy source of pre-generated grid files you can obviously skip this section.

A grid file is generated from a fit file using the program make_grid:

>make_grid 20001123.kap.fit > 20001123.kap.grd

The program writes the grid file to the standard output, so the pipe ">" is used to redirect it to the file "20001123.kap.grd". The index file associated with the fit file is not required for this kind of processing.

To monitor what is going on you can use the verbose option:

>make_grid -vb 20001123.kap.fit > 20001123.kap.grd

In the default mode make_grid produces records that contain two minutes of data, this can be over-ridden using "-i" option and specify the record length in seconds:

>make_grid -vb -i 180 20001123.kap.fit > 20001123.kap.grd

The program uses the scan flag contained in the fit file to identify individual Radar scans, but you will often find that the scan flag is incorrectly set and make_grid will fail. To overcome this problem you can specify a fixed scan length using the "-tl" option:

>make_grid -vb -tl 120 20001123.kap.fit > 20001123.kap.grd

In this mode, data is read from the file in two-minute chunks, synchronized with the start of the day. So all the data between 0:00UT and 0:02UT is assumed to come from the first scan, and so on.

It is important to distinguish the difference between the "-i" and "-tl" options as they can have a dramatic impact on the output produced. The processing consists of two stages, the first involves filtering whole radar scans to remove noise and ground scatter contamination. The "-tl" option sets the length of each input scan, and at the end of the filtering process the scan will consist of filtered, averaged data with no more than one velocity measurement in each range/beam cell. The second stage involves fitting the filtered data to an equi-area grid. Filtered scans are mapped to grid for the specified record length as defined by the "-i" option. If the scan interval is less than the record length, multiple scans will form an output record. If a grid cell contains multiple velocities these are averaged in the output record.

There are many more command line options that can be applied to make_grid, the most useful allow you to specify the start and end time of the period to process:

>make_grid -vb -st 9:00 -et 13:00 -tl 120 -i 120 20001123.kap.fit > 20001123.kap.grd

Generating GrdMap Files

The next generation of radar software uses a new self-describing format called "dmap" for storing gridded data. To distinguish the new format data from the old the suffix "grdmap" is used.

The command line option "-new" is used to create a file using the GrdMap format:

>make_grid -new 20001123.kap.fitacf > 20001123.kap.grdmap

Note that with the "-new" option the input files must also be in the new dmap based fitacf format.

A grid file can be converted into a grdmap file using the utility gridtogrdmap:

>gridtogrdmap 20001123.kap.grd > 20001123.kap.grdmap

You can also convert a grdmap file into a grid file using the utility grdmaptogrid:

>grdmaptogrid 20001123.kap.grdmap > 20001123.kap.grd

Using cFit files

Grid and GrdMap files can also be created from cFit format data. The cFit format is a highly compressed version of the fitted ACF data.

The option "-cfit" indicates that the input file is a cFit format file:

>make_grid -new -cfit 20001123.kap.cfit > 20001123.kap.grdmap

Combining Grid and GrdMap Files

A grid or grdmap file with data from only one Radar site is not particularly useful for generating global convection maps. The next step of the process is to combine the data from individual radars together into a single file. This is done using the program combine_grid:

>combine_grid 20001123.pyk.grd 20001123.han.grd 20001123.gbr.grd > 20001123.grd

Again note that the program writes the file to standard output and so must be redirected into a file. One obvious short cut is to use wildcards:

>combine_grid 20001123.???.grd > 20001123.grd

One thing to be careful of is that your wildcarded name doesn't match the name of the output file, or you'll end up with a recursion and a really big output file. In the case above the "?" wildcard was used rather than "*" for this very reason.

You can combine together a set of grid files and you can combine together a set of grdmap files, but you cannot combine together a mixture of the two.

The option "-new" indicates that the input files are in the grdmap format:

>combine_grid -new 20001123.???.grdmap > 20001123.grdmap

The only other option that combine_grid takes is "-r", which indicates that if two of the input files contain data from the same radar, only the data from the file last on the command line will be included in the output. This is a confusing way of saying that you can use this option to replace data from one file with another. This is particularly useful if you discover that you need to re-process the data from one radar and don't have access to the individual grid files:

>combine_grid -r 20001123.grd 20001123.pyk.grd.new > 20001123.grd.new