Provides FDSN Web Services
fdsnws is a server that provides FDSN Web Services from a SeisComP3 database and record source. The following services are available:
Service | Retrieves this... | In this format |
---|---|---|
fdsnws-dataselect | time series data in miniSEED format | miniSEED |
fdsnws-station | network, station, channel, response metadata | FDSN Station XML, StationXML, SC3ML |
fdsnws-event | contributed earthquake origin and magnitude estimates | QuakeML, SC3ML |
The available services can be reached from the fdsnws start page. The services also provide an interactive URL builder providing the URL based on the selection.
If fdsnws is started, it accepts connections by default on port 8080 which can be changed in the configuration. Also please read Changing the service port for running the services on a privileged port, e.g. port 80 as requested by the specification.
Note
If you decide to run the service on a different URL than localhost:8080 you have to change the URL string in the *.wadl documents located under $DATADIR/fdsnws.
http://localhost:8080/fdsnws/dataselect/1/query?net=GE&sta=BKNI&cha=BH?&start=2013-04-11T00:00:00&end=2013-04-11T12:00:00
To summit POST request the command line tool curl may be used.
sysop@host:~$ curl -X POST --data-binary @request.txt "http://localhost:8080/fdsnws/dataselect/1/query"
The data streams exposed by this service may be restrict by defining an inventory filter, see section Filtering the inventory.
The inventory exposed by this service may be restricted, see section Filtering the inventory.
The streams served by the Station and DataSelect service may be filtered by specified an INI file in the stationFilter and dataSelectFilter configuration parameter. You may use the same file for both services or define a separate configuration set. Note: If distinct file names are specified and both services are activated, the inventory is loaded twice which will increase the memory consumption of this module.
[Chile]
code = CX.*.*.*
[!Exclude station APE]
code = GE.APE.*.*
[German (not restricted)]
code = GE.*.*.*
restricted = false
shared = true
archive = GFZ
The listing above shows a configuration example which includes all Chile stations. Also all not restricted German stations, with exception of the station GE.APE, are included.
The configuration is divided into several rules. The rule name is given in square brackets. A name starting with an exclamation mark defines an exclude rule, else the rule is an include. The rule name is not evaluated by the application but is plotted when debugging the rule set, see configuration parameter debugFilter.
Each rule consists of a set of attributes. The first and mandatory attribute is code which defines a regular expression for the stream code (network, station, location, channel). In addition the following optional attributes exist:
Attribute | Type | Network | Station | Location | Channel |
---|---|---|---|---|---|
restricted | Boolean | X | X | X | |
shared | Boolean | X | X | X | |
netClass | String | X | |||
archive | String | X | X |
A rule matches if all of its attributes match. The optional attributes are evaluated bottom-up where ever they are applicable. E.g. if a rule defines restricted = false but the restricted flag is not present on channel level then it is searched on station and then on network level. If no restricted attribute is found in the hierarchy, the rule will not match even if the value was set to false.
The individual rules are evaluated in order of their definition. The processing stops once a matching rule is found and the stream is included or excluded immediately. So the order of the rules is important.
One may decided to specify a pure whitelist, a pure blacklist, or to mix include and exclude rules. If neither a matching include nor exclude rule is found, then channel is only added if no other include rule exists in the entire rule set.
The FDSN Web service specification defines that the Service SHOULD be available under port 80. Typically SeisComP3 runs under a user without root permissions and therefore is not allowed to bind to privileged ports (<1024). To serve on port 80 you may for instance
authbind allows a program which does not or should not run as root to bind to low-numbered ports in a controlled way. Please refer to man authbind for program descriptions. The following lines show how to install and setup authbind for the user sysop under the Ubuntu OS.
sysop@host:~$ sudo apt-get install authbind
sysop@host:~$ sudo touch /etc/authbind/byport/80
sysop@host:~$ sudo chown sysop /etc/authbind/byport/80
sysop@host:~$ sudo chmod 500 /etc/authbind/byport/80
Once authbind is configured correctly the FDSN Web services may be started as follows:
sysop@host:~$ authbind --deep seiscomp exec fdsnws
In order use authbind when starting fdsnws as SeisComP service the last line in the ~/seiscomp3/etc/init/fdsnws.py have to be commented in.
All major Linux distributions ship with their own firewall implementations which are front-ends for the iptables kernel functions. The following line temporary adds a firewall rule which redirects all incoming traffic on port 8080 to port 80.
sysop@host:~$ sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to 8080
Please refer to the documentation of your particular firewall solution on how to set up this rule permanently.
The FDSNWS standard requires HTTP digest authentication as the authentication mechanism. The "htpasswd" configuration option is used to define the location of the file storing usernames and passwords of users who are allowed to get restricted data. Any user with valid credentials would have access to all restricted data.
An extension to the FDSNWS protocol has been developed in order to use email-pattern-based access control lists, which is an established authorization mechanism in SC3 (used by Arclink). It works as follows:
The authentication extension is enabled by setting the "auth.enable" configuration option to "true" and pointing "auth.gnupgHome" to a directory where GPG stores its files. Let's use the directory ~/seiscomp3/var/lib/gpg, which is the default.
sysop@host:~$ mkdir -m 700 ~/seiscomp3/var/lib/gpg
sysop@host:~$ gpg --homedir ~/seiscomp3/var/lib/gpg --gen-key
sysop@host:~$ gpg --homedir ~/seiscomp3/var/lib/gpg --import <keys.asc
sysop@host:~$ gpg --homedir ~/seiscomp3/var/lib/gpg --edit-key XXXXXXXX sign save
sysop@host:~$ echo "auth.enable = true" >>~/seiscomp3/etc/fdsnws.cfg
A client like fdsnws_fetch is recommended, but also tools like wget and curl can be used. As an example, let's request data from the restricted station AAI (assuming that we are authorized to get data of this station).
sysop@host:~$ wget --post-file token.asc https://geofon.gfz-potsdam.de/fdsnws/dataselect/1/auth -O cred.txt
sysop@host:~$ curl --data-binary @token.asc https://geofon.gfz-potsdam.de/fdsnws/dataselect/1/auth -o cred.txt
sysop@host:~$ wget "http://`cat cred.txt`@geofon.gfz-potsdam.de/fdsnws/dataselect/1/queryauth?starttime=2015-12-15T16:00:00Z&endtime=2015-12-15T16:10:00Z&network=IA&station=AAI" -O data.mseed
sysop@host:~$ curl --digest "http://`cat cred.txt`@geofon.gfz-potsdam.de/fdsnws/dataselect/1/queryauth?starttime=2015-12-15T16:00:00Z&endtime=2015-12-15T16:10:00Z&network=IA&station=AAI" -o data.mseed
sysop@host:~$ fdsnws_fetch -a token.asc -s 2015-12-15T16:00:00Z -e 2015-12-15T16:10:00Z -N IA -S AAI -o data.mseed
fdsnws inherits global options.
Type: IP
Defines the bind address of the server. "0.0.0.0" allows any interface to connect to this server whereas "127.0.0.0" only allows connections from localhost. Default is 0.0.0.0.
Type: int
Server port to listen for incoming requests. Note: The FDSN Web service specification defines the service port 80. Please refer to the documentation on how to serve on privileged ports. Default is 8080.
Type: int
Number of maximum simultaneous requests. Default is 5.
Type: int
Maximum number of objects per query, used in fdsnws-station and fdsnws-event to limit main memory consumption. Default is 10000.
Type: int
Restricts end time of requests to current time - realtimeGap seconds. Negative values allowed. Used in fdsnws-dataselect. WARNING: If this value is unset and a realtime recordsource (e.g. slink) is used, requests may block if end time in future is requested.
Type: float
Maximum number of samples (in units of million) per query, used in fdsnws-dataselect to prevent a single user to block one connection with a large request.
Type: string
Path to password file used in fdsnws-station/queryauth. The format is 'username:password' separated by lines. Because of the HTTP digest authentication method required by the FDSN specification, the passwords have to be stored in plain text. Default is @CONFIGDIR@/fdsnws.htpasswd.
Type: string
Path to access log file. If unset no access log is created.
Type: boolean
Enables/disables access to restricted inventory data. Default is true.
Type: boolean
If enabled removes author information from any event creationInfo element. Default is false.
Type: string
If set the event service will only return events having a preferred origin with a matching evaluationMode property.
Type: boolean
Enables/disables the DataSelect service. Default is true.
Type: boolean
Enables/disables the Event service. Default is true.
Type: boolean
Enables/disables the Station service. Default is true.
Type: string
Path to station inventory filter file.
Type: string
Path to dataselect inventory filter file.
Type: boolean
If enabled a debug line is written for each stream ID explaining why a stream was added/removed by a inventory filter. Default is false.
Type: string
Defines the prefix for the default filenames if downloading and saving data from within a browser. For data loaded using dataselect, it is thus fdsnws.mseed by default. Default is fdsnws.
Type: list:string
List of enabled event types
Type: list:string
List of disabled event types
Type: boolean
Save request log to database. Default is false.
Type: string
Default user. Default is fdsnws.
Type: boolean
Enable auth extension. Default is false.
Type: string
GnuPG home directory. Default is @ROOTDIR@/var/lib/gpg.
show help message.
show version information
Use alternative configuration file. When this option is used the loading of all stages is disabled. Only the given configuration file is parsed and used. To use another name for the configuration create a symbolic link of the application or copy it, eg scautopick -> scautopick2.
Load given plugins.
Run as daemon. This means the application will fork itself and doesn't need to be started with &.
Enable/disable self-shutdown because a master module shutdown. This only works when messaging is enabled and the master module sends a shutdown message (enabled with --start-stop-msg for the master module).
Sets the name of the master-module used for auto-shutdown. This is the application name of the module actually started. If symlinks are used then it is the name of the symlinked application.
Sets the name of the master-username of the messaging used for auto-shutdown. If "shutdown-master-module" is given as well this parameter is ignored.
Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug
Increase verbosity level (may be repeated, eg. -vv)
Quiet mode: no logging output
Limits the logging to a certain component. This option can be given more than once.
Use syslog logging back end. The output usually goes to /var/lib/messages.
Path to lock file.
Send log output to stdout.
Debug mode: --verbosity=4 --console=1
Use alternative log file.
List all supported database drivers.
The database connection string, format: service://user:pwd@host/database. "service" is the name of the database driver which can be queried with "--db-driver-list".
The configmodule to use.
Load the inventory from the given database or file, format: [service://]location
Do not use the database at all
List all supported record stream drivers
The recordstream source URL, format: [service://]location[#type]. "service" is the name of the recordstream driver which can be queried with "--record-driver-list". If "service" is not given "file://" is used.
Specify a file as record source.
Specify a type for the records being read.