Help: fedcatalog v.1

Description

The irisws-fedcatalog web service returns selected time series channels from across multiple FDSN data centers. The combined catalog of channels is updated daily from each contributing data center.

This interface is primarily designed for discovery of data channels and subsequent requesting of time series using FDSN Web Service interfaces. The service is also capable of removing overlap from the channel list when the same channels are available from multiple data centers, avoiding the request and processing of duplicate data.

This interface is designed to support client-side federated data access.

FDSN Data Centers

A list of FDSN data centers (returns JSON) contributing to the combined catalog is available in JSON format and includes web service endpoints at each data center.

For data centers that wish to have their center included in the IRIS Federator system see the new data centers section.

Design and use philosophy

This interface is designed to support client-side federated data access. In this model, user software submits a request to the federator catalog and then submits the request(s) identified by the catalog service to the appropriate data centers.

The irisws-fedcatalog interface accepts requests as they would be submitted to an fdsnws-station or fdsnws-dataselect web service. The channel list returned for a query is in a form ready to be submitted to either of those interfaces (the POST form). The web service end points at each data center to which the request(s) should be submitted are also provided in the response by this service. As this interface and what it returns to the user is modeled on FDSN web services it is relatively easy to adapt for any users or software already requesting data through FDSN web services.

Metadata as a proxy for data availability

Because the catalog of channels is based on the station metadata published by each data center, and the metadata does not contain a full accounting of data availability, there is no guarantee of data availability for every match found by the catalog. For example, a data recording outage of a few hours or days (or even longer) is usually not reflected in the station metadata.

Removing overlapping channel entries

In some cases the same data are available from multiple data centers. By default, irisws-fedcatalog will remove any overlapping channel entries matching the submitted criteria. This behavior can be controlled using the includeoverlaps parameter. Overlapping entries are removed according to business rules that identify the primary (based on network) or best data center for any given channel.

Data retrieval methods

To retrieve data from the fedcatalog service, submit a request by either of the methods below:

  • via HTTP GET : Provide a series of parameter-value pairs in the URL that specify the start-time and end-time, along with the desired network(s), station(s), location(s) and channel(s). Wildcards are supported. Please visit the irisws-fedcatalog service interface for parameter usage details.

  • via HTTP POST: Submit a request to the service containing a list of the desired networks, stations, locations, channels, start-times and end-times. Detailed examples are shown at the bottom of this page.
  • via web service clients: Dedicated clients that use the irisws-fedcatalog service can be found on the Client list.

Output formats

In all output formats metadata holdings are organized by data center. If the request matches channel entries from multiple data centers, the data centers are listed alphabetically. All parameters listed under the “service-options” or labeled “constraints” are applied as search filters to the metadata holdings. In contrast, parameters labeled as “passive options” are simply passed through the service and returned in the request format response only.

The default request format (…&format=request)

The request output is a simple text format designed to allow a client to identify data centers (and their service endpoints) along with pre-formatted requests ready to be submitted to each data center.

The response always begins with a repeat of the search criteria and pass-through options that are not time series or time window specific. This is followed by sections specific to each data center offering channels matching the criteria.

Each data center section begins with the DATACENTER keyword, followed by *SERVICE keywords identifying service endpoints, which is followed by time series and time window identifiers in the POST request format required by standard fdsnws-station and fdnsws-dataselect services. Note that in the request output format the time windows of the metadata holdings are trimmed to the time constraints of the user’s request. Data center sections are separated by empty lines.

The template for the request format is:

[parameter=value]
[parameter=value]

DATACENTER=<name>,<Website URL>
STATIONSERVICE=<URL to fdsnws-station service>
DATASELECTSERVICE=<URL to fdsnws-dataselect service>
*SERVICE=<URL to other services>
Net Sta Loc Chan StartTime EndTime
Net Sta Loc Chan StartTime EndTime
Net Sta Loc Chan StartTime EndTime
...

DATACENTER=<name>,<Website URL>
STATIONSERVICE=<URL to fdsnws-station service>
DATASELECTSERVICE=<URL to fdsnws-dataselect service>
*SERVICE=<URL to other services>
Net Sta Loc Chan StartTime EndTime
Net Sta Loc Chan StartTime EndTime
...

  • Note: not all data centers support all service types, a minimum level of support are the fdsnws-station (STATIONSERVICE) and fdsnws-dataselect (DATASELECTSERVICE) services. Other entries currently used are EVENTSERVICE, RESPSERVICE and SACPZSERVICE, mostly as a convenience. A complete list of data centers and endpoints reported by this system is available in JSON format.

The text format (…&format=text)

The text output is identical to the text format defined in the fdsnws-station service specification. The amount of detail in the text output of course varies according to the specified level setting. The entries for each data center are in separate sections identified with comments. Unlike the request output type, text output does not trim the start and end dates of the metadata holdings to the user’s time constraints.

New data centers

For data centers that would like have their center included in the IRIS Federator system please make your request via email and include links to your FDSN web services. A few requirements must be met for inclusion in the Federator:

  • Data center must have operational fdsnws-station and fdsnws-dataselect services
  • The fdsnws-station service must support text output format (format=text parameter)
  • The fdsnws-dataselect service must offer some openly available (non-restricted) data
  • The services must be publicly reachable, continuously available and stable
  • Not a requirement, but ideally the fdsnws-station service should support the matchtimeseries option
  • Not a requirement, but ideally the fdsnws-station service should support the includeavailability option to return at least data availability extents.

Example queries and responses

This is a simple query for a single channel (IU.ANMO.00.BHZ):

/irisws/fedcatalog/1/query?net=IU&sta=ANMO&loc=00&cha=BHZ&starttime=2000-01-01&endtime=2002-01-01

and the subsequent response:

DATACENTER=IRISDMC,http://ds.iris.edu
RESPSERVICE=http://service.iris.edu/irisws/resp/1/
EVENTSERVICE=http://service.iris.edu/fdsnws/event/1/
STATIONSERVICE=http://service.iris.edu/fdsnws/station/1/
DATASELECTSERVICE=http://service.iris.edu/fdsnws/dataselect/1/
SACPZSERVICE=http://service.iris.edu/irisws/sacpz/1/
IU ANMO 00 BHZ 2000-01-01T00:00:00 2000-10-19T16:00:00
IU ANMO 00 BHZ 2000-10-19T16:00:00 2002-01-01T00:00:00

  • Note the identification of the DATACENTER, the service endpoints at that data center followed by a (short) list of channels for that data center.
  • Note the inclusion of two time ranges (epochs) for this channel. This is a reflection of the time ranges in the metadata reported by the data center.

A query for BHZ channels within a certain radius from Hawaii:

/irisws/fedcatalog/1/query?latitude=19.6&longitude=-155.5&maxradius=5&channel=BHZ&starttime=2000-01-01&endtime=2000-02-01&nodata=404

lat=19.6
lon=-155.5
maxradius=5.0

DATACENTER=GEOFON,http://geofon.gfz-potsdam.de
STATIONSERVICE=http://geofon.gfz-potsdam.de/fdsnws/station/1/
DATASELECTSERVICE=http://geofon.gfz-potsdam.de/fdsnws/dataselect/1/
GE MAUI -- BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00
ZF KAHU -- BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00
ZF LANI -- BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00
ZF LAUP -- BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00
ZF PUNA -- BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00

DATACENTER=IRISDMC,http://ds.iris.edu
RESPSERVICE=http://service.iris.edu/irisws/resp/1/
EVENTSERVICE=http://service.iris.edu/fdsnws/event/1/
STATIONSERVICE=http://service.iris.edu/fdsnws/station/1/
DATASELECTSERVICE=http://service.iris.edu/fdsnws/dataselect/1/
SACPZSERVICE=http://service.iris.edu/irisws/sacpz/1/
G KIP -- BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00
IU KIP 00 BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00
IU KIP 10 BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00
IU POHA 00 BHZ 2000-01-01T00:00:00 2000-02-01T00:00:00

  • Note the identification of two data centers and their service endpoints. Also, the passing-through of the geographic search criteria which the caller may or may not choose to include in subsequent queries to the data centers.

Example of a POST query

To submit a more complex request with a list of network, station, location, channel and time ranges a request may be POSTed to the service query interface (http://service.iris.edu/irisws/fedcatalog/1/query).

For example, the following shows the contents of a local file called fedcatalog.request:

$ cat fedcatalog.request
TA A25A -- BH? 2010-03-25T00:00:00 2010-04-01T00:00:00
IU ANMO * BH? 2010-03-25T00:00:00 2010-04-01T00:00:00
IU ANMO 10 HHZ 2010-03-25T00:00:00 2010-04-01T00:00:00
II KURK 00 BH? 2010-03-25T00:00:00 2010-04-01T00:00:00

This could then be submitted using common, command-line looks such as wget or curl:

wget --post-file=fedcatalog.request -O fedcatalog.txt http://service.iris.edu/irisws/fedcatalog/1/query
curl -L  --data-binary @fedcatalog.request -o fedcatalog.txt http://service.iris.edu/irisws/fedcatalog/1/query

We recommend always using the -L option to allow curl to follow HTTP redirections specified by our systems. The DMC uses HTTP redirection during maintenance to keep servicing requests.


Problems with this service?

Please send an email report of which service you were using, your URL query, and any error feedback to:
data-help@earthscope.org
We will address your issue as soon as possible.