Wiki » History » Version 22
Version 21 (Chad Trabant, 01/07/2014 04:57 PM) → Version 22/38 (Chad Trabant, 01/07/2014 06:01 PM)
h1. Ringserver
Ringserver is a generic ring buffer. The server is commonly used as a buffer for (near) real-time streaming feeds of time series data. Each individual time series is usually identified with a stream name (for seismological data these are equivalent to unique channels). The stream names are completely arbitrary allowing conventions to be created for specific needs.
In general, data clients connect to the server and submit a "subscription" for specific streams and as other data clients submit data into the system they are sent to clients with matching subscriptions. By identifying each buffer packet uniquely and allowing a client to select a starting point in the buffer, clients can resume broken connections without data loss.
Communicating with a server is generally done using the DataLink protocol over TCP/IP. A DataLink client can query the server for various server details, submit data, and subscribe to streams. Server details can include stream lists, client lists and server statistics. A library in C that implements the DataLink protocol is available.
The dalitool program can be used to monitor and diagnose a ringserver node. Source code for this program and the ringserver itself is available here:
http://www.iris.edu/pub/programs/ringserver
h1. Server configuration, on-the-fly changes
The server can be configured using either command line options or a configuration file.
Most options, if specified via the configuration file, can be changed on the fly.
[[Configuring the server]]
[[ringserver manual]]
[[ringserver configuration file]] with detailed descriptions of options
h1. Usage logging
The server can optionally write transmission logs to record all data sent from the server or submitted to the server. Commonly CHAD
h1. SeedLink support
While the architecture implemented by ringserver is completely generic some additions have been made to support the "SeedLink":http://www.iris.edu/data/dmc-seedlink.htm protocol. In particular for packets where the data are 512-byte miniSEED records the server can be configured to serve the streams via SeedLink. This option must be enabled, listening for SeedLink connections is not done by by default.
h1. Stand-alone SeedLink server
The ringserver software may be used to implement a stand-alone SeedLink server. One pre-requisite to serving data via SeedLink from ringserver to is have data formatted as 512-byte miniSEED data records. To accomplish this there are two options:
* Users that already have 512-byte miniSEED written to files may configure ringserver to continuously scan directories and files and read any newly arriving data. Read [[How to configure ringserver as a SeedLink streaming server]] for more information.
* Users that can write a program that can submit 512-byte miniSEED data to ringserver via DataLink (TCP/IP) simply need to configure the server to listen for SeedLink connections and run their submission client.
h1. Implementation and resource requirements
The ring buffer managed by ringserver is a traditional first-in-first-out (FIFO) buffer, once the buffer is full adding new data packets pushes the oldest data packets out of the buffer. The buffer can be run in any of these configurations:
* (Default) As a memory-mapped file. This allows the buffer to be larger than the available memory and be stateful. Default size is 1 Gigabyte and is only limited to machine resources, buffer sizes above 2 Gigabytes will generally require a 64-bit binary.
* Read from a file into memory on startup and written to the same file on shutdown. Allows for a stateful buffer and for situations where memory-mapping is not possible or practical.
* As a volatile memory-only buffer. Used in special scenarios only, the buffer is lost on shutdown.
h1. 3rd party libraries used
The ringserver program uses the following open source libraries:
* PCRE - Perl Compatible Regular Expressions (www.pcre.org)
* Mini-XML - Lightweight XML Library (www.minixml.org)
Ringserver is a generic ring buffer. The server is commonly used as a buffer for (near) real-time streaming feeds of time series data. Each individual time series is usually identified with a stream name (for seismological data these are equivalent to unique channels). The stream names are completely arbitrary allowing conventions to be created for specific needs.
In general, data clients connect to the server and submit a "subscription" for specific streams and as other data clients submit data into the system they are sent to clients with matching subscriptions. By identifying each buffer packet uniquely and allowing a client to select a starting point in the buffer, clients can resume broken connections without data loss.
Communicating with a server is generally done using the DataLink protocol over TCP/IP. A DataLink client can query the server for various server details, submit data, and subscribe to streams. Server details can include stream lists, client lists and server statistics. A library in C that implements the DataLink protocol is available.
The dalitool program can be used to monitor and diagnose a ringserver node. Source code for this program and the ringserver itself is available here:
http://www.iris.edu/pub/programs/ringserver
h1. Server configuration, on-the-fly changes
The server can be configured using either command line options or a configuration file.
Most options, if specified via the configuration file, can be changed on the fly.
[[Configuring the server]]
[[ringserver manual]]
[[ringserver configuration file]] with detailed descriptions of options
h1. Usage logging
The server can optionally write transmission logs to record all data sent from the server or submitted to the server. Commonly CHAD
h1. SeedLink support
While the architecture implemented by ringserver is completely generic some additions have been made to support the "SeedLink":http://www.iris.edu/data/dmc-seedlink.htm protocol. In particular for packets where the data are 512-byte miniSEED records the server can be configured to serve the streams via SeedLink. This option must be enabled, listening for SeedLink connections is not done by by default.
h1. Stand-alone SeedLink server
The ringserver software may be used to implement a stand-alone SeedLink server. One pre-requisite to serving data via SeedLink from ringserver to is have data formatted as 512-byte miniSEED data records. To accomplish this there are two options:
* Users that already have 512-byte miniSEED written to files may configure ringserver to continuously scan directories and files and read any newly arriving data. Read [[How to configure ringserver as a SeedLink streaming server]] for more information.
* Users that can write a program that can submit 512-byte miniSEED data to ringserver via DataLink (TCP/IP) simply need to configure the server to listen for SeedLink connections and run their submission client.
h1. Implementation and resource requirements
The ring buffer managed by ringserver is a traditional first-in-first-out (FIFO) buffer, once the buffer is full adding new data packets pushes the oldest data packets out of the buffer. The buffer can be run in any of these configurations:
* (Default) As a memory-mapped file. This allows the buffer to be larger than the available memory and be stateful. Default size is 1 Gigabyte and is only limited to machine resources, buffer sizes above 2 Gigabytes will generally require a 64-bit binary.
* Read from a file into memory on startup and written to the same file on shutdown. Allows for a stateful buffer and for situations where memory-mapping is not possible or practical.
* As a volatile memory-only buffer. Used in special scenarios only, the buffer is lost on shutdown.
h1. 3rd party libraries used
The ringserver program uses the following open source libraries:
* PCRE - Perl Compatible Regular Expressions (www.pcre.org)
* Mini-XML - Lightweight XML Library (www.minixml.org)