Feature #607
Replace hard coded list of supported data types with list provided in -service.cfg file
0%
Description
Currently, the supported data types, such as text, xml, miniseed, etc. are restricted to types coded into WSS. This feature is to replace the selection of one "outputType" with a list of pairs of outputType to HTTP content type.
The WSS configuration files will continue to load using Java Properties, this means the new form of outputType would look something like
outputType = \
xml: application/xml \
text: text/plain \
miniseed: application/vnd.fdsn.mseed \
mseed: application/vnd.fdsn.mseed
There is an additional implication, that only miniseed data will use Seisfile parsing to determine data usage, otherwise, data will be copied directly to the client without processing.
History
#1 Updated by Mike Stults almost 10 years ago
ready for test, this changed caused a few ripples in how types are handled
in the service.cfg file, the outputType parameter is replaced with
- outputTypes - specifies a list of "key: mediaType" pairs accepted by this
- service. The "mediaType is used to set the HTTP Content-Type response
- header element for each query. For any query, the default mediaType can
- be overridden by adding the parameter "&format=otherKey", where otherKey
- is one of the keys in the outputTypes list. #
- outputTypes is optional - the default is "binary: application/octet-stream".
- When outTypes is specified in this file, the first pair in the list
- overrides the default. #
- items may be on separate lines, use a backslash as described for java
- property files. For each pair,
- - Each pair of items must be separated with a comma
- - The key must be separated from its mediaType with a colon
- - Keys and mediaType may not contain spaces
- - For miniseed usage log processing, the list must include the pair
- miniseed: application/vnd.fdsn.mseed
#
outputTypes = \
miniseed: application/vnd.fdsn.mseed, \
mseed: application/vnd.fdsn.mseed, \
json: application/json, \
text: text/plain,\
texttree: text/plain,\
xml: application/xml
- The mediaType for a respective query is passed through to the response header as-is. So the configuration can define types as needed.
- The new default type is binary: application/octet-stream, not text as before
- response header content-type will be "attachment" for miniseed, mseed, and binary only, otherwise "inline"
response header content-type filename suffix will be the respective "key" as defined in outputTypes, except no suffix for binary type.
- The first pair in the list is the default type.
#2 Updated by Chad Trabant almost 10 years ago
Testing with :
When outputTypes is not configured
- default of application/octet-stream works
- requesting format=mseed results in 400 error and somewhat confusing error message & typo.
- Content-Disposition filename has no extension
When outputTypes=mseed:application/vnd.fdsn.mseed
- default is application/vnd.fdsn.mseed
- format=mseed works as expected
- format=miniseed result in 400 error as expected
- Content-Disposition filename has .mseed extension
When outputTypes=mseed:application/vnd.fdsn.mseed miniseed:application/vnd.fdsn.mseed
- Configuration error as expected
When outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed
- default is application/vnd.fdsn.mseed with .mseed filename extension
- format=mseed works and results in .mseed filename extension
- format=miniseed works and results in .miniseed filename extension
In all cases, when format is an allowed parameter, format=binary is accepted by the shell.
When format=mseed or format=miniseed the usage log contains _summary and per-channel log entries as expected.
In conclusion: the new outputTypes functionality is working as expected in all test cases above.
#3 Updated by Chad Trabant almost 10 years ago
- Status changed from New to In Progress
One change suggested. When an unacceptable format is submitted by the user (e.g. format=bogusformat
), the WSS returns such an error:
Error 400: error from ParameterTranslator, excp: WebServiceShell, unrecognized outpTtype key: BOGUSFORMAT from input: bogusformat must be a type in outputTypes in ...service.cfg
For the service log this is OK (note the typo of outpTtype
), but it's too bulky to be clear for the end user.
I suggest returning something more like this:
Error 400: Unrecognized format value: BOGUSFORMAT
which is much more directly related to what the user supplied.
#4 Updated by Mike Stults almost 10 years ago
updated messaging and pushed new snapshot to nexus, message now reads like this
Error 400: Error with WebServiceShell parameter, Unrecognized format value: BOGUSFORMAT the value must be in the outputTypes list in service.cfg
#5 Updated by Chad Trabant almost 10 years ago
Yeah, that's a bit better, but 2/3 of the text is still unimportant to the user, plus the important bit is sandwiched between two parts that only a developer or operator knows about and has control over.
Error 400: Error with WebServiceShell parameter, Unrecognized format value: BOGUSFORMAT
the value must be in the outputTypes list in service.cfg
I would really rather: put the highlighted part first, drop the "the value must be in the outputTypes list in service.cfg" altogether (or relegate it to the service log file only) and move the WSS parameter part to the end (in brackets or something) or a different line if possible.
If we need different levels of logging details for operators/developers versus users let's talk about that.
#6 Updated by Robert Weekly almost 10 years ago
Following up on the new 1.1.9-SNAPSHOT.war and another round of testing...
1. Specifying outputTypes=mseed:application/vnd.fdsn.mseed and format=mseed
Content-Disposition: attachment; filename=fdsnws_dataselect_DATETIME.mseed
Content-Type: application/vnd.fdsn.mseed
2. Specifying outputTypes=mseed:application/vnd.fdsn.mseed and format=miniseed
Content-Type: text/plain
Error 400: Error with WebServiceShell parameter, Unrecognized format value: miniseed the value must be in the outputTypes list in service.cfg
3. Specifying outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed and format=mseed
Content-Disposition: attachment; filename=fdsnws_dataselect_DATETIME.mseed
Content-Type: application/vnd.fdsn.mseed
4. Specifying outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed and format=miniseed
Content-Disposition: attachment; filename=fdsnws_dataselect_DATETIME.miniseed
Content-Type: application/vnd.fdsn.mseed
5. Specifying outputTypes=mseed:application/vnd.fdsn.mseed miniseed:application/vnd.fdsn.mseed and format=miniseed
Error 500: Application Configuration Issue
This error does not communicate any information about what part of the configuration is causing problems.
When I use curl to POST, I get a slightly different error...despite having postEnabled=true
in my dataselect service configuration file.
Error 400: POST Method not allowed
6. Specifying outputTypes=mseed:application/vnd.fdsn.mseed,miniseed:application/vnd.fdsn.mseed and format=BADFORM_123
Error 400: Error with WebServiceShell parameter, Unrecognized format value: BADFORM_123 the value must be in the outputTypes list in service.cfg
In short, the outputTypes parameter appears to be returning the correct content for the HTTP header and accurate error messages, EXCEPT in test case 5 where the outputTypes parameter is incorrectly defined using a space between the key-value lists instead of a comma.
#7 Updated by Mike Stults almost 10 years ago
Mad error messages shorter, now only prepended with "Wss - "
#8 Updated by Mike Stults over 9 years ago
- Status changed from In Progress to Closed
Complete and tested