Project

General

Profile

Feature #607

Replace hard coded list of supported data types with list provided in -service.cfg file

Added by Mike Stults over 7 years ago. Updated about 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Start date:
10/01/2014
Due date:
% Done:

0%

Estimated time:
Resolution:

Description

Currently, the supported data types, such as text, xml, miniseed, etc. are restricted to types coded into WSS. This feature is to replace the selection of one "outputType" with a list of pairs of outputType to HTTP content type.

The WSS configuration files will continue to load using Java Properties, this means the new form of outputType would look something like

outputType = \
xml: application/xml \
text: text/plain \
miniseed: application/vnd.fdsn.mseed \
mseed: application/vnd.fdsn.mseed

There is an additional implication, that only miniseed data will use Seisfile parsing to determine data usage, otherwise, data will be copied directly to the client without processing.

History

#1 Updated by Mike Stults over 7 years ago

ready for test, this changed caused a few ripples in how types are handled

in the service.cfg file, the outputType parameter is replaced with

  1. outputTypes - specifies a list of "key: mediaType" pairs accepted by this
  2. service. The "mediaType is used to set the HTTP Content-Type response
  3. header element for each query. For any query, the default mediaType can
  4. be overridden by adding the parameter "&format=otherKey", where otherKey
  5. is one of the keys in the outputTypes list. #
  6. outputTypes is optional - the default is "binary: application/octet-stream".
  7. When outTypes is specified in this file, the first pair in the list
  8. overrides the default. #
  9. items may be on separate lines, use a backslash as described for java
  10. property files. For each pair,
  11. - Each pair of items must be separated with a comma
  12. - The key must be separated from its mediaType with a colon
  13. - Keys and mediaType may not contain spaces
  14. - For miniseed usage log processing, the list must include the pair
  15. miniseed: application/vnd.fdsn.mseed #
    outputTypes = \
    miniseed: application/vnd.fdsn.mseed, \
    mseed: application/vnd.fdsn.mseed, \
    json: application/json, \
    text: text/plain,\
    texttree: text/plain,\
    xml: application/xml

- The mediaType for a respective query is passed through to the response header as-is. So the configuration can define types as needed.
- The new default type is binary: application/octet-stream, not text as before
- response header content-type will be "attachment" for miniseed, mseed, and binary only, otherwise "inline"
response header content-type filename suffix will be the respective "key" as defined in outputTypes, except no suffix for binary type.
- The first pair in the list is the default type.

#2 Updated by Chad Trabant over 7 years ago

Testing with :

When outputTypes is not configured

  • default of application/octet-stream works
  • requesting format=mseed results in 400 error and somewhat confusing error message & typo.
  • Content-Disposition filename has no extension

When outputTypes=mseed:application/vnd.fdsn.mseed

  • default is application/vnd.fdsn.mseed
  • format=mseed works as expected
  • format=miniseed result in 400 error as expected
  • Content-Disposition filename has .mseed extension

When outputTypes=mseed:application/vnd.fdsn.mseed miniseed:application/vnd.fdsn.mseed

  • Configuration error as expected

When outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed

  • default is application/vnd.fdsn.mseed with .mseed filename extension
  • format=mseed works and results in .mseed filename extension
  • format=miniseed works and results in .miniseed filename extension

In all cases, when format is an allowed parameter, format=binary is accepted by the shell.

When format=mseed or format=miniseed the usage log contains _summary and per-channel log entries as expected.

In conclusion: the new outputTypes functionality is working as expected in all test cases above.

#3 Updated by Chad Trabant over 7 years ago

  • Status changed from New to In Progress

One change suggested. When an unacceptable format is submitted by the user (e.g. format=bogusformat), the WSS returns such an error:

Error 400: error from ParameterTranslator, excp: WebServiceShell, unrecognized outpTtype key: BOGUSFORMAT  from input: bogusformat  must be a type in outputTypes in ...service.cfg

For the service log this is OK (note the typo of outpTtype), but it's too bulky to be clear for the end user.

I suggest returning something more like this:

Error 400: Unrecognized format value: BOGUSFORMAT

which is much more directly related to what the user supplied.

#4 Updated by Mike Stults over 7 years ago

updated messaging and pushed new snapshot to nexus, message now reads like this

Error 400: Error with WebServiceShell parameter, Unrecognized format value: BOGUSFORMAT the value must be in the outputTypes list in service.cfg

#5 Updated by Chad Trabant over 7 years ago

Yeah, that's a bit better, but 2/3 of the text is still unimportant to the user, plus the important bit is sandwiched between two parts that only a developer or operator knows about and has control over.

Error 400: Error with WebServiceShell parameter, Unrecognized format value: BOGUSFORMAT the value must be in the outputTypes list in service.cfg

I would really rather: put the highlighted part first, drop the "the value must be in the outputTypes list in service.cfg" altogether (or relegate it to the service log file only) and move the WSS parameter part to the end (in brackets or something) or a different line if possible.

If we need different levels of logging details for operators/developers versus users let's talk about that.

#6 Updated by Robert Weekly over 7 years ago

Following up on the new 1.1.9-SNAPSHOT.war and another round of testing...

1. Specifying outputTypes=mseed:application/vnd.fdsn.mseed and format=mseed

Content-Disposition: attachment; filename=fdsnws_dataselect_DATETIME.mseed
Content-Type: application/vnd.fdsn.mseed

2. Specifying outputTypes=mseed:application/vnd.fdsn.mseed and format=miniseed

Content-Type: text/plain
Error 400: Error with WebServiceShell parameter, Unrecognized format value: miniseed the value must be in the outputTypes list in service.cfg

3. Specifying outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed and format=mseed

Content-Disposition: attachment; filename=fdsnws_dataselect_DATETIME.mseed
Content-Type: application/vnd.fdsn.mseed

4. Specifying outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed and format=miniseed

Content-Disposition: attachment; filename=fdsnws_dataselect_DATETIME.miniseed
Content-Type: application/vnd.fdsn.mseed

5. Specifying outputTypes=mseed:application/vnd.fdsn.mseed miniseed:application/vnd.fdsn.mseed and format=miniseed

Error 500: Application Configuration Issue

This error does not communicate any information about what part of the configuration is causing problems.

When I use curl to POST, I get a slightly different error...despite having postEnabled=true in my dataselect service configuration file.

Error 400: POST Method not allowed

6. Specifying outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed and format=BADFORM_123

Error 400: Error with WebServiceShell parameter, Unrecognized format value: BADFORM_123 the value must be in the outputTypes list in service.cfg

In short, the outputTypes parameter appears to be returning the correct content for the HTTP header and accurate error messages, EXCEPT in test case 5 where the outputTypes parameter is incorrectly defined using a space between the key-value lists instead of a comma.

#7 Updated by Mike Stults about 7 years ago

Mad error messages shorter, now only prepended with "Wss - "

#8 Updated by Mike Stults about 7 years ago

  • Status changed from In Progress to Closed

Complete and tested

Also available in: Atom PDF