Commit | Line | Data |
4536f655 |
1 | |
2 | Stem::LogTail Design Notes |
3 | |
4 | The Stem::LogTail module provides a service similar to the standard Unix |
5 | program tail -f. It constructs a Cell that can watch a file for changes |
6 | in it size or inode and then sends the new data to a destination. The |
7 | destination is a Logical Log created elsewhere in the Stem application |
8 | and it can be located on any Hub on the network. Also file status |
9 | changes such as not being found, first time it is opened, it was |
10 | truncated, etc. can be sent to a different Logical Log than the data. |
11 | This module is a critical part of the StemLog application which |
12 | transfers active log files from one system to another. |
13 | |
14 | The action of checking the file is triggered by a call to the tail_cmd |
15 | method. There are two main ways of triggering it, either by an internal |
16 | timer or via a command message directed at this Cell. The timer is |
17 | configured when the Cell is created and its resolution in in seconds. So |
18 | e.g. you can cause a file check to occur every 15 seconds. If a 'tail' |
19 | command message is sent to this Cell, it will also trigger a file |
20 | check. The command message is usually sent from a Stem::Cron entry |
21 | configured elsewhere. In either case new data is sent to the 'data_log' |
22 | Log and status is sent to the 'status_log' Log. |
23 | |
24 | The primary configuration parameter is the path to the file to be |
25 | checked. Also the required 'data_log', optional 'status_log' and |
26 | 'repeat_interval' are set in the configuration. |
27 | |
28 | When the file is checked, the current size and inode are compared to the |
29 | previous values. If either has changed, then either data and/or status log |
30 | entries are sent. |
31 | |
32 | NOTE: The modification time of a file is not currently checked for |
33 | changes. The reason is that there is no way to determine if a file |
34 | changed (or what data has changed) if the file just has its modification |
35 | time changed (by touch or utime). This means that there is a weakness in |
36 | just checking for the file size changing. If a file were to be truncated |
37 | and then written to with the same amount of data as it had when it was |
38 | last checked, a tail operation would not report any new data. This bug |
39 | exists in both the GNU and Solaris tail programs. The only way around |
40 | this is to check for modification time changes and trust |
41 | this. Stem::LogTail will support an option for doing this instead of |
42 | checking file size but it is up to the user to assure that the file will |
43 | only change its modification time if its data actually changes. Most |
44 | logs generated by programs have this attribute. |