NETTAB File Format Description
Introduction
This document describes the second generation of the nettab format, introduced in 2012, and used internally at GEOFON since then. The nettab format describes seismological station metadata information in plain text files. The description can be split into different files that contain lines of various types as defined below. Each line begins with a two-character type specifier, followed by containing the information.
While the tools do not enforce any behaviour other than one network per file, it is advised to keep the network tree and instruments in separate files. This way we would have a file for each network being described, a file with instruments and its poles and zeros. There are optional files that describes default attributes that will be more explained in the context of the tab2inv tool, used to convert nettabv2 set of files to SeisComP3 XML inventory.
Once this tool (tab2inv) starts processing a new file, the attributes defined in the previous network file are not valid anymore. That is, attributes are valid for the current file only. The attributes defined on the defaults file are valid for processing all other tab files. The nettab format is also used by tabinvmodifier to modify existing SeisComP metadata.
General comments about this document
In this document fields that are shown between square brackets ('[' and ']') are optional. Fields that are shown between curly brackets ('{' and '}') have their own sub-format that is explained in this document.
1. How to write tab files
Tab files are simple ASCII files. For example, this file:
Nw: IQ 1980/001 Na: Description="Plate Boundary Project Iquique, GFZ Potsdam, Germany" Sa: Restricted=True * *,*,* Sl: UNAP "Uni-Iquique/Chile" DM24%A1383 CMG-3ESP/60%T34622 100 ZNE -20.24393 -70.14041 0.0 0.0 2009/134 Sl: NEUQ "Neuquen-mine/Chile" DM24%C617 CMG-3ESP/60%T34639 100 ZNE -20.17220 -70.07323 1043.0 0.0 2009/137
would define a network with code IQ with description as indicated by the Na line, having two stations (UNAP and NEUQ) each, with three components (ZNE) with 100 samples per second with high gain channels (HHZ. HHN, HHE). Each valid line of a tab file starts with a two letter line identifier and a colon (':').
Lines starting with '#' are not processed, and blank lines are allowed
# This is a ignored comment line
Any Start/End date values should be represented by the format:
YYYY/JJJ[:HHMM]
where JJJ is the day of the year e.g. 001 for January 1, 224 for September 1 (except in leap years).
Valid line identifiers are of three types:
2. Network-to-Channels related lines are used to describe the channel tree (the NSLC hierarchy)
- Nw: Network Line
- Na: Network Attribute Line (applies to network or station group)
- Sl: Station Line
- Sa: Station Attribute Line (applies to station, location or channels)
3. Instruments lines are used to define responses.
- Se: Seismometer Line
- Dl: Datalogger Line
- Cl: Calibration Line
- Ff: FIR Filter
- If: IIR Filter
- Pz: Pole & Zeros Filter
- Ia: Instrument Attribute Line (applies to sensor, datalogger, calibration, FIR filter, IIR filter, and poles & zeros filters)
4. Station Group related lines allow virtual networks to be defined.
- Sg: Station Group Line
- Sr: Station Group Reference Line
These are described in turn in the following three sections.
2. Network-to-channel identifiers
Nw
The Network Header (Nw) starts a new network. It normally is located at the beginning of the file (above all Sl and Sa lines).
Nw: Code {Start} [{End}]
Na
The Network Attribute line (Na) defines one attribute (like a variable) for the current Network, defined by the last (Nw) line. Each attribute would be mapped to one network, for all the period of operation of the network. If you want to change an attribute of a network node you should define a new network with the same code and different start and end times (without overlapping the old one) and set the parameters again applying the desired changes to the appropriate ones.
The general format for the (Na) line is:
Na: KEY=VALUE
KEY is actually any string other than (Code, Start and End) that should be attributed to the (Nw) line. Normally you would like to specify the attributes as defined by the SeisComp3 inventory since up to now the only converter that exists converts from nettab to sc3inventory, but other values like WWW, ContactPerson or any other can be specified. Those when not mapped to the Sc3 inventory will not be integrated to the Sc3 schema but other tools could make use of them, while ignoring the sc3 mapped ones.
The VALUE field is the value that this key would assume. When the VALUE has spaces it is necessary to place the string inside double quotes like this:
Na: Description="Special quoted network description"
NOTE: The attribute names used in a tab file begin with a capital letter, e.g., 'Archive', 'NetClass'. (In SeisComp3 inventory XML they have lower case or camel case.)
Sa
The station attribute line (Sa) adds attributes to the station, location or channel level just like the (Na) does to networks. The main difference is that the (Sa) lines has to be present before the station are defined while the (Na) applies to an already defined network. In the case that you change a Sa line in the middle of a file, the second definition would overwrite the value of the first definitions but only, for the stations defined by the Sl lines that are defined below the position of the second Sa line for the defined KEY. For example
Nw: GE 1993/001 Na: Description="GEOFON network" Na: Region="World" Sl: SAMI ... Sa: Description="SAMI station on GE network" SAMI
it would not work because the station line (Sl) for the station SAMI appears before the station attribute line for this station. The correct order is:
Nw: GE 1993/001 Na: Description="Ge network" Na: Region="World" Sa: Description="SAMI station on GE network" SAMI Sl: SAMI ...
The general syntax for the station attribute line is:
Sa: KEY=VALUE {Element[s]} [to={Date}] [from={Date}]
Where KEY and VALUE are like in the network attribute (Na) but noticing that none of the values specified directly in a station line (Sl) could be overwritten by attributes. Also, each station attribute line can be applied to one or more elements according to the element patterns given. The element definition is a pattern including up to 3 fields matching stations, location and channels always separated by commas (','). A pattern without commas matches stations only while a pattern with one comma matches a location only and a pattern with two commas matches channels.
Some examples are:
* Matches any station *,* Matches any location *,*,* Matches any channel APE,,* Matches all the channels of the empty location code from station APE
Patterns can be built using the normal wildcard characters * and ?. It is only important to note that each pattern applies only to one node of the instrument generated, so, if you want to apply a restricted flag to a station APE and channel BHZ
Sa: Restricted=True APE Sa: Restricted=True APE,*,BHZ
or in one line:
Sa: Restricted=True APE APE,*,BHZ
Also, each variable is applicable only to a determined time range, defined using the 'to=' and 'from=' options. The dates supplied in the to and from field should match exactly a time span defined by the station lines (Sl).
Sl
The station line (Sl) is the line that defines the streams (channels) for each station and its location(s) in a compact form. For each epoch of the station you can define a new line for the same network code. Also, to define two different location codes or channel gains for the same station it is also necessary to define different lines with the same Code.
The general form of the station line is:
Sl: Code {Region} {Datalogger} {Sensor} {Channels} {Orientation} Latitude Longitude Elevation Depth {Start} [{End}]
Each of the displayed fields above has its own "sub-format" as explained below:
Code is the station code, normally a sequence of up to five letter or numbers that identifies the station.
Region is the combination of the place and country in the following form:
"Place[/Country]"
For example:
"Tibet Plateau/Tibet"
Note that the double quotes are needed to keep the spaces so that they are not lost in the conversion.
The Datalogger and Sensor fields are actually KEYS to the instruments which are defined by sensor (Se) and datalogger (Dl) lines. As explained below, normally those keys are associated with the datalogger and sensor models to make the file more readable. Also, if appropriate this is the place to insert the serial number of the equipment used for those channels and the total gain of datalogger or sensor if different from the default as supplied in sensor line (Se) or datalogger line (Dl).
The general syntax is:
KEY[%SerialNumber[%TotalGain]] KEY[%SerialNumber[%TotalGain]]
Some examples are:
Q380-M%QT930604 ## key=Q380-M and serial number=QT930604 FBA-EST/10.0%yyyy ## key=FBA-EST/10.0 and serial number=yyyy (unset) PS6-HG%xxxx%10000000.0 ## key=PS6-HG, serial xxxx (unset) and gain=10000000.0
Please note that since it is impossible to define the gain without a proper serial number the value xxxx is expected for dataloggers and the value yyyy is expected for sensors representing unset values.
Channels is the most important part of the station line (Sl). This is where you specify which sampling rate, location and gain code are used in the channels generated by this line. The format of this field is:
F[1|2]/L??/T[H|L]_[LetterModifier]SamplingRate/[.../[LetterModifier]SamplingRate]
The channel specification is composed of two parts, the preamble and a main part. These are separated by an underscore, '_'.
A channel without preamble would default to Location code empty, high gain ('H' as the middle letter on the gain channel naming) and compression format equal to 2 / i.e. Steim2. The letters F, L and T in the preamble can be used to change those values. i.e. L10 sets location code to 10 while TL sets the gain to L. The main part of the Channels definition defines the samples rates available. Each sample rate is a number optionally prepended by a letter. When not supplied the following translation table between sample rates and codes will apply.
Code | Sample Rate |
H >= | 80 |
S >= | 40 |
B > | 1 |
L == | 1 |
V == | 0.1 |
U == | 0.01 |
Following the Channels fields comes the Orientation field. This defines the last letter of the channel codes. In this field up to three orientations can be defined by supplying one of the standard letters (ZNE12) or, the letter followed by the dip and azimuth pairs like in:
OneCharCode[(Dip,Azimuth)]
Here OneCharCode can be any character but for one of the special characters defined by the table below it is not necessary to supply the dip and azimuth since they are pre-defined as:
Character | Dip | Azimuth | Overwrite |
Z | -90.0 | 0.0 | no |
N | 0.0 | 0.0 | no |
E | 0.0 | 90.0 | no |
1 | 0.0 | 0.0 | yes |
2 | 0.0 | 90.0 | yes |
For the codes values of 12 you are able to overwrite the dip/azimuth definition. If you supply one of the standards ZNE you should not supply a dip/azimuth. And if you supply anything other than 12ZNE you should give the dip and azimuth directions.
Latitude is the latitude coordinate of the channels ranging from -90 to 90 degrees.
Longitude is the longitude coordinate of the channels ranging from -180 to 180 degrees.
Altitude is the altitude of the channel in meters.
Depth is the depth (borehole) of the seismometer positive from the surface downwards (opposite as altitude).
Start and optionally End are the dates used for the channel definition. One station line should not overlap the other station line in time. Lines for different gains or sampling rates can overlap each other since the channel code generated will be different.
Since the Sl line is responsible to generate tree levels of the NSLC tree each of then with different Start and End times, but with the constrains that the Channels should be contained in the Locations and Locations contained in the Stations, the parsing algorithm while adding channels inside the stations will adjust the Start and End Times of the Station and Location nodes as appropriated. While extending the operation periods of Station and Location it will compare the attributes indicated for each level to confirm if the new element parsed in the current line fits a previous Sl line element already created in memory (and in inventory) or, we should create a new epoch. If you want, you can overwrite the dates auto generated using Sa lines.
Some examples of lines are:
- To generate three channels (HHZ, HHN, HHE) for station MN15 with a Trident & Trillium (datalogger and sensor) located in Morocco.
Sl: MM15 "Emintanoute/Morocco" Trident/N%2448 Trillium-120%969 100 ZNE 31.1991 -8.8973 955.0 0.5 2010/321 2011/078
- If this station had maintenance on March 19, 2011 ("2011/078") and the Trident with serial number "2448" was replaced by the equipment "2327" one would provide two Sl lines in the tab file:
Sl: MM15 "Emintanoute/Morocco" Trident/N%2448 Trillium-120%969 100 ZNE 31.1991 -8.8973 955.0 0.5 2010/321 2011/078 Sl: MM15 "Emintanoute/Morocco" Trident/N%2696 Trillium-120%969 100 ZNE 31.1991 -8.8973 955.0 0.5 2011/096 2012/116:1219
to represent this change.
- A more complicated example, like for station KBS in the GEOFON network we would have something like:
## First Epoch (Two sensors, sts-1 (20,1,0.1 sps) and sts-2 (80/40 sps) Sl: KBS "Ny Alesund/Spitsbergen" Q380-IU%xxxx STS-1/VBB%S1kbs1 F2_20/1/0.1 ZNE 78.9256 11.9417 77.0 3.0 1994/309 1999/105 Sl: KBS "Ny Alesund/Spitsbergen" Q380-IU%xxxx STS-2/N%yyyy F2_E80/40 ZNE 78.9256 11.9417 77.0 3.0 1994/309 1999/105 ## Second Epoch (each sensor in a different location code new 40 and 1 sps in location 10) Sl: KBS "Ny Alesund/Spitsbergen" Q380-IM%xxxx STS-1/VBB%S1kbs1 F2/L00_20/1/0.1 ZNE 78.9256 11.9417 77.0 3.0 1999/105 2010/158 Sl: KBS "Ny Alesund/Spitsbergen" Q380-IM%xxxx STS-2/N%yyyy F2/L10_80/B40/1 ZNE 78.9256 11.9417 77.0 3.0 1999/105 2010/158 ## Third Epoch (accelerometer FBA, 80 now is 100 sps in location 10 and location 10 has new 0.1 sps) ## channel orientation adjusted from ZNE to Z12 Sl: KBS "Ny Alesund/Spitsbergen" Q330/HR%xxxx STS-1/VBB%S1kbs1 F2/L00_20/1/0.1 Z12 78.9256 11.9417 77.0 3.0 2010/159 Sl: KBS "Ny Alesund/Spitsbergen" Q330/HR%xxxx STS-2/N%yyyy F2/L10_100/B40/1/0.1 Z12 78.9256 11.9417 77.0 3.0 2010/159 Sl: KBS "Ny Alesund/Spitsbergen" Q330/N%xxxx FBA-EST/10.0%yyyy F2/L20/TN_100/1 Z12 78.9256 11.9417 77.0 3.0 2010/159
3. Instrument identifiers
As described above the datalogger and sensor parameters given on a Sl line are references to instruments, normally loaded from a different file that is used from all other nettab files describing the stations. We normally call this file inst.db and it is a normal nettab file that contains the set of lines that describe the instruments.
Ia
The Ia lines are similar to the Na and Sa lines in terms of function. Those lines define extra attributes to any of the valid instruments lines defined below (as the Sa line attributes Ia lines should be defined before the Instrument definition). The general syntax of an Ia line is:
Ia: KEY=VALUE {Type}::Name [... {Type}::Name]
The Type is actually a line specifier type (Se, Dl, Cl, Ff, Pz or If) and Name, the instrument name or any of the allowed wildcards ('*' or '?'). The Type will fix the KEY to certain instrument type and is normally used together with the wildcards to apply attributes to all instruments of a the same type.
For example:
Ia: Type="BB" Se::*
would set all Sensor types to Broad-Band (BB) independent of their names since we are using the '*' as name.
Se
Sensor lines specify a Seismometer or one Accelerometer instrument. The general form of this line is:
Se: Name Gain gFrequency Normalization nFrequency noz nop Zeros [...] Poles [...]
where the Name is actually the key that will be used to link this instrument to the third column of the Sl line as described above. The Name identifier should be unique and normally it is expected to resemble the instrument name and company. The other parameters that describe the instrument needed to complete the line are:
Gain is the default gain for the seismometer. This is the gain that will be used if, on the Sl line, no gain is supplied.
gFrequency is the frequency that the gain was measured, should be applied.
Normalization is the normalization factor of the seismometer.
nFrequency is the frequency at which this normalization should be applied.
noz is an integer indicating the number of zeros included in the response.
nop is an integer indicating the number of poles included in the response.
Zeros and Poles are complex numbers optionally followed by a repetition index. The complex number format is (x,y) and the repetition value should be given without spaces line in 2(0.0,0.0). This is quite handy for specifying the Zeros that are normally all equal. The parser expect to find noz + nop complex numbers specified.
Example: The specification for LE seismometer used by our current inst.db file is:
Se: LE-3D/1 400.0 1.0 1.4142 1.0 2 2 2(0.0,0.0) (-4.4429,4.4429) (-4.4429,-4.4429)
Dl
Datalogger lines specify a Datalogger instrument. The general form of this line is:
Dl: {Id} {Gain} {Spfr} {Mcld} [{FilterIndex} {Stages}]
Id - string, label for referring to this datalogger, used in station lines (Sl:).
Gain - float
Spfr - numeric
Mcld - float
These can be followed by
- FilterIndex - string? Refers to a?? Use "None" to...
- and a Stages string, providing a list of decimations. TODO.
Examples:
Dl: TITAN-3XT 429000.0 2000.0 0.0 TITAN 62.5_1/2/3/4/5 Dl: DM24 392157.0 256000.0 0.0 DM24 100_1/2/3/4/5/6/7,50_1/2/3/4/5/6/7/9,40_1/2/3/4/5/6/11,25_1/2/3/4/5/6/7/10,20_1/2/3/4/5/6/7/8,1 Dl: Q380-U 410000.0 5120.0 0.0 Q380U 80_1/2,40_1/2/3,20_1/2/3/4,1_1/2/3/4/5/6,0.1_1/2/3/4/5/6/9 Dl: LS-7000 409165.0 100.0 0.0 None 100,20 Dl: CMG-SAM 1.0e6 100.0 0.0 None 100
Cl
These lines specify a sensor or datalogger calibration element in the inventory. The calibration overrides the gain given in the inventory by a calibrated value for all components of the instrument. The general form of this line is:
Cl: {SerialNumber} {Gain Channel 1} {Gain Channel 2} {Gain Channel 3} {Instrument ID list}
{SerialNumber}: Is the serial number of the instrument that was calibrated. {Gain Channel 1|2|3}: Are the gains values calibrated for the instrument {Instrument ID list}: Are keys separated by commas to the Se or Dl lines defining the Original Manufacturer/Model of the Instrument
Example:
# Single channel calibrations - STS-1 Cl: S1isp1 2550.0 2324.0 2292.0 S_STS-1/VBB Cl: S1kbs1 2596.0 2336.0 2400.0 S_STS-1/VBB Cl: S1kmbo1 2452.0 2312.0 2332.0 S_STS-1/VBB [...]
Ff
These lines specify a FIR filter definition. Filters are defined in a separate filters directory. The general form of this line is, from right to left:
Ff: {Id} {Filename} {sym} {ncf} 0 {inrate} {fac} {delay} {corrtn} {gain} {frg}
Id - string, label
Filename - name of a file in the filters directory
sym - symmetry code, letter 'B', 'C', or...
ncf - integer, number of coefficients
nullo - float, not used, historical?? TODO
inrate - numeric
fac - int
delay - float, gets multiplied by inrate.
corrtn - float, gets multiplied by inrate.
gain - float
frg - float
Example:
# FIR filter list for Quanterra Q330 digitizer and Seiscomp recorder # Name Sym ncf inrate fac delay corrtn gain frg Ff: Q330_FIR_1 q330_b100_100 A 65 0 100.0 1 0.041607 0.041607 1.0 0.0 Ff: Q330_FIR_2 q330_b100_50 A 81 0 50.0 1 0.531607 0.531607 1.0 0.0 [...]
The filters directory contains a file called q330_b100_100 having 65 lines:
0 1.315493e-11 0.000000e+00 1 1.501065e-04 0.000000e+00 2 1.339681e-02 0.000000e+00 [...] 62 1.056275e-08 0.000000e+00 63 2.577911e-09 0.000000e+00 64 -7.018623e-10 0.000000e+00
and other similar files, one for each name in an Ff: line. These define the filter response functions.
Pz
Poles and zeroes, for IIR filter on a datalogger. Constructed with pzType = 'D'. TODO
Pz: {Id} {Gain} {GainFrequency} {NormalizationFactor} {NormalizationFrequency} {noz} {nop} {Poles-and-Zeros}
Id - string, label.
Gain - float.
GainFrequency - float.
NormalizationFactor - float.
NormalizationFrequency - float.
noz - integer, number of zeroes.
nop - integer, number of poles.
Then from the end... Poles-and-Zeroes object is a list of nop+noz tuples of points in the complex plane. These are similar to Se: lines described above.
Example:
# Digitizer analog response list Pz: HDR24_digipaz_1 0.538 1.0 1.40631E+12 1.0 0 3 (-9904.8,3786.0) (-9904.8,-3786.0) (-12507.3,0.0)
If
Poles and zeroes, for digiPaz filter on a datalogger. These lines have the same format as a Pz: line, but are constructed with pzType = 'A'. TODO
4. Station Group
These define a "virtual network", that references a list of stations assigned to different real networks which should be accessible under a single network code (a virtual network code). By convention, virtual network codes start with an underscore "_". Since the stations defined in this process are only keys to real networks while building XML from virtual network tab files can only be performed as a final step of loading the inventory.
Sg
This line is like the Nw line for a "real" network. As for Nw, {End} is optional, and if the end date for the station group is left unset it will be absent in inventory. The Sg line may be followed by as many Na lines as needed to define the attributes of the station group, just like in a normal network definition case. The general expected format for the Sg line is:
Sg: {Virtual network code} {Start} [{End}]
For example,
Sg: _SOAM 2011/001
Defines a station group, with code _SOAM assembled from 2011/001. It is a common practice to use underscore as the first character of a virtual network.
Sr
A station reference defines a linked reference from a station defined in the inventory to the current Virtual Network. The general format of the Sr line is:
Sr: {Netcode},{Stationcode} [from={end} to={date}]
Indicates that the station {Stationcode}, listed under network code {Netcode}, should be included in the current station group. There must be one of these lines for each station included in the station group. The optional from= and to= attributes can be used to set start and end dates to filter certain epochs of stations to be linked.
Example: in a (hypothetical) _OST.tab file, we have a group of seven stations:
Sg: _OST 2014/001 2014/365 Na: Description="Ostpreussen Seismisches Netzwerk" Na: Type=VBB Sr: CZ,NKC Sr: GE,HLG Sr: GE,MORC Sr: GE,RUE Sr: GE,RGN Sr: PL,GKP Sr: PL,KSP
It is important to understand that defining a Virtual Network in a Tab file is not sufficient for obtaining a XML description of the virtual network using the tab2inv tool. Since Sr lines defines references they need to be rebuilt while the XML inventory is generated. This is done by the tab2inv command when it have access to the inventory XML files pre-loaded in the system or access to the database, in those cases, it will during the conversion resolve the stations references.
Keeping this in mind, it is necessary to first convert all the Nw tab files from TAB -> SC3INV XML, import other dataless files as desired and in a later stage only perform the conversion of the virtual networks, Sg+Sr lines, to SC3 Inventory XML. This will guarantee that all references are rebuild correctly in the system. Building the virtual networks reference at conversion time gives a maximum flexibility to user that can integrate into the virtual network, stations defined in different source other than the tab files itself.