Project

General

Profile

Support #854

Ringserver configuration

Added by Catherine Pequegnat about 7 years ago. Updated about 7 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Start date:
12/14/2016
Due date:
% Done:

0%

Estimated time:
Resolution:

Description

Hello,
we are running ringserver 2015.188 on CentOS Linux release 7.2.1511 (Core) and our configuration has the following MSeedScan option :

MSeedScan /home/sysop/work/out StateFile=/home/sysop/scan.state InitCurrentState=y

which we understand as :
ringserver process will start using the information store in /home/sysop/scan.state and will not rescan all the files
in /home/sysop/work/out

But if we shutdown, then restart ringserver, all the files within /home/sysop/work/out are systematically and fully rescanned.

Do you have any suggestion we could follow to avoid that ?

Thanks a lot,
Best regards

Catherine Péquegnat
RESIF datacentre

History

#1 Updated by Chad Trabant about 7 years ago

Hi Catherine,

The InitCurrentState option is only in effect in these two cases:

       If  the  InitCurrentState  option  is set to ’y’ the scanning will only
       read new data, effectively skipping all the data discovered during  the
       first scan, under the following conditions:
       1) No StateFile has been specified
       2) StateFile has been specified but does not exist

So with the MSeedScan configuration you specified two scenarios are possible:

  1. If `/home/sysop/scan.state` does not exist on startup then do not read current data on first scan through all files, new data added to the files after the first scan will be read and ingested into ringserver.
  2. If `/home/sysop/scan.state` does exist on startup then start reading data from files and offsets recorded in the state file. New files not included in the state file will be read in full.

In all cases, a scan through all files needs to be conducted so the scanner knows what is present. At a minimum this does a directory walk and stat(2) on each file. It does not read all the data from each file if instructed to skip it, because either a) InitCurrentState is in effect or b) the state file indicates some portion has already been read.

Are you observing that, when you restart ringserver, the data that was read into the ringserver before the restart is read into ringserver again? So duplicate data is going to clients? If so, I would guess there are problems writing and/or reading the statefile. Of course, there could also be a bug.

Chad

#2 Updated by Catherine Pequegnat about 7 years ago

Hi Chad,
thanks for your explaination and your help
I did some tests and noticed the following :
- when stopping a ringserver driven by "metasys" script, the file scan.state is emptied ... but is still existing. So, in my configuration ( MSeedScan /home/sysop/work/out StateFile=/home/sysop/scan.state InitCurrentState=y) scan.state exists and InitCurrentState is not used; mseed files are rescanned and data is going to clients... :-(
- when stopping a ringserver by kill -9, the file scan.state is udpdated, and the restart does not send data again.
Conclusion : if I want to use metasys, the correct configuration is : MSeedScan /home/sysop/work/out InitCurrentState=y

Cheers

Catherine

#3 Updated by Chad Trabant about 7 years ago

Hi Catherine,

Something is definitely not working correctly. The state file should be written on shutdown and not be empty if data was scanned. Also, a kill -9 should not make the server write the state file, in fact it does not allow the server to write the file because it never has the opportunity. A kill -9 will also likely result in corrupt buffers.

I did some testing here with version 2015.188 and found the exact opposite results. The scan state file was written correctly on shutdown (Control-C). The scan state was not written on kill -9.

Perhaps the metasys log file has some interesting details, such as the process being killed before shutting down cleanly?

If you do not care about pulling in all the data in a scanned location and only want clients to get the current, latest data then your Conclusion solution (with no state file) should work.

cheers,
Chad

Also available in: Atom PDF