Commit | Line | Data |
4536f655 |
1 | |
2 | Demonstration of Log Tail |
3 | |
4 | This demonstration script illustrates Stem's ability to monitor log |
5 | files. It showcases the Stem modules Stem::Log (which logs messages) |
6 | and Stem::LogTail (which checks files for changes/updates). This |
7 | demonstration runs two hubs named archive and monitor. The monitor |
8 | hub watches a particular log file for changes. When a change occurs, |
9 | messages are sent to the archive hub to be logged. The archive |
10 | hub records the contents of the monitored log file (sent by the |
11 | monitor hub) and also records status messages sent by the monitor |
12 | hub. Log messages that are recorded by the archive hub can be |
13 | stored as either raw data or with custom formats. This demonstrates |
14 | a single log file being monitored, in a real world case there could be |
15 | several log files being monitored. It is easy to see in this example that |
16 | Stem can handle this with a small number of additions to its |
17 | configuration files. This can be distributed securely across a network. |
18 | This demo script is described in detail below with sections on |
19 | running, using, configuring, and experimenting with it. |
20 | |
21 | Running tail_demo |
22 | |
23 | The log tail demonstration is called tail_demo and it uses monitor.stem |
24 | and archive.stem configuration files (found in conf/). It is run with |
25 | the simple command: |
26 | |
27 | tail_demo |
28 | |
29 | To exit, just enter 'q' in the tail_demo script window itself. It will |
30 | kill all the other windows and then exit. This will also happen if you |
31 | interrupt the demo script with ^C. |
32 | |
33 | If the -s option is used, then all the windows will use ssfe (split |
34 | screen front end) which provides a command line edit and history window |
35 | section and an output section. Two hub windows named Archive and |
36 | Monitor will be created and a single shell window will be created with its |
37 | current directory set to tail/. Stem will create two log files |
38 | in the tail/ directory, bar.log and bar_status.log. bar.log is used |
39 | by the archive hub to record what ever is sent to that log file and |
40 | bar_status.log is used as a log file for status messages. |
41 | The hub windows can be used to interact with that hubs Stem environment |
42 | and the command line window can be used to put contents into a foo.log |
43 | file. The two hub windows use the standard module Stem::TtyMsg which |
44 | allows you to interact with them. In this demo they will be used to |
45 | modify the Stem environment which will affect the behavior of the |
46 | logical logs. |
47 | |
48 | Using tail_demo |
49 | |
50 | Initially, bar.log and bar_status.log will be empty files but in 10 |
51 | seconds (the tailing interval set in the monitor.stem configuration) |
52 | the status log will have a message about foo.log not being found. Run |
53 | ls -l several times in the shell (center) window to see when the status |
54 | has been logged and then read that file will |
55 | |
56 | $ cat bar_status.log |
57 | |
58 | Now type the following at the command line (in the shell window): |
59 | |
60 | $ echo 'foobar' > foo.log |
61 | |
62 | After 10 seconds (configured in the Stem configuration file) |
63 | you can look in bar.log and you will notice that there is a single line |
64 | that reads, "foobar", and in bar_status.log you will notice that there |
65 | is a status message saying that it is the first time that foo.log was |
66 | opened. And, of course, we have the line "foobar" in the monitored log |
67 | file, foo.log. |
68 | |
69 | Configuring tail_demo |
70 | |
71 | Look at the file conf/monitor.stem. That is one of the configuration files |
72 | used by tail_demo. It is very simple and easy to understand. It is a Perl list |
73 | of lists structure with key/value pairs. Read the config_notes for more |
74 | on this. |
75 | |
76 | The first Cell configured is Stem::Hub which names this hub as |
77 | 'monitor'. |
78 | |
79 | [ |
80 | class => 'Stem::Hub', |
81 | name => 'monitor', |
82 | args => [], |
83 | ], |
84 | |
85 | Next comes the configuration for the Stem::Portal cell, |
86 | |
87 | [ |
88 | class => 'Stem::Portal', |
89 | args => [ |
90 | ], |
91 | ], |
92 | |
93 | It is important to note here that there are no args passed into the |
94 | portal. This means that the portal is a client portal, its default |
95 | host is set to localhost, and its default port is set to 10,000. For |
96 | more information on portals, read the portal design notes. |
97 | |
98 | The next Cell configured is Stem::TtyMsg which supports typing in and |
99 | sending command messages. This is used in all the demo configurations. |
100 | |
101 | [ |
102 | class => 'Stem::TtyMsg', |
103 | args => [], |
104 | ], |
105 | |
106 | The next cell is the application specific part in the monitor.stem |
107 | configuration, the Cell configuration for Stem::LogTail: |
108 | |
109 | [ |
110 | 'class' => 'Stem::LogTail', |
111 | 'name' => 'foo', |
112 | 'args' => [ |
113 | 'path' => 'tail/foo.log', |
114 | 'repeat_interval' => 10, |
115 | 'data_log' => 'archive:bar', |
116 | 'status_log' => 'archive:bar_status', |
117 | ], |
118 | ], |
119 | |
120 | This is the cell responsible for monitoring the indicated log file |
121 | (foo.log). It has arguments for the path to the monitored file, what |
122 | the time interval is (in seconds) to check the file for changes, the |
123 | data log, and the status log. Note that the address of the data and status |
124 | log indicates both the name of the hub that it is located at, as well as, the |
125 | name of the log cell. For more information on these configuration options |
126 | please see the tail log design notes. |
127 | |
128 | Now, lets take a look at the conf/archive.stem configuration file. This |
129 | file defines logical log files (bar and bar_status) that are used by the |
130 | monitor.stem configuration file to log the changes to foo.log. |
131 | |
132 | The first three cells that are configured are the same as the monitor |
133 | configuration, Stem::Hub, Stem::Portal, and Stem::TtyMsg. They are |
134 | for the most part identical. It is worth mentioning here that the |
135 | configuration for the Portal has its server boolean flag set to true, |
136 | indicating that this portal will be awaiting connection requests from |
137 | remote Portals anywhere on a network. |
138 | |
139 | The next three are the configurations for Stem::Log logical logs. The |
140 | first one is a typical logical log file configuration, |
141 | |
142 | [ |
143 | 'class' => 'Stem::Log', |
144 | 'args' => [ |
145 | 'name' => 'bar', |
146 | 'path' => 'tail/bar.log', |
147 | 'filters' => [ |
148 | file => 1, |
149 | forward => [ 'bar_stdout' ], |
150 | ], |
151 | ], |
152 | ], |
153 | |
154 | This is defining a logical log named "bar" that is associated with |
155 | a real log file indicated by the path, "tail/bar.log". It also |
156 | has a filters argument that allows Stem::Log::Entries to be filtered |
157 | before they are placed in the log file. The first of the filter |
158 | operations in the above configuration, 'file', indicates that the |
159 | incoming log entry should be placed in the file indicated by the 'path' |
160 | argument. Another one of these filter rules, 'forward, indicates that |
161 | the log entry is always forwarded to the log 'bar_stdout'. |
162 | |
163 | |
164 | The next Stem::Log configuration defines a logical log that conditionally |
165 | outputs its entries to STDOUT. It will write log file entries to STDOUT |
166 | if the 'bar_stdout' Stem environment variable is set to be greater than |
167 | the severity level of the log entry. |
168 | |
169 | [ |
170 | 'class' => 'Stem::Log', |
171 | 'args' => [ |
172 | |
173 | 'name' => 'bar_stdout', |
174 | 'format' => '%f [%L][%l] %T', |
175 | 'strftime' => '%D %T', |
176 | 'filters' => [ |
177 | 'env_gt_level' => 'bar_stdout', |
178 | stdout => 1, |
179 | ], |
180 | ], |
181 | ], |
182 | |
183 | This configuration specifies a format (overriding the default raw format |
184 | like bar.log) that displays the entry with timestamps, label, and level. |
185 | This allows you to customize your log entries to your liking. |
186 | To demonstrate the severity level detection, do the following at the |
187 | Stem prompt in the archive hub window (upper left), |
188 | |
189 | bar_stdout=8 |
190 | |
191 | This will ensure that if the severity level of the incoming log entry |
192 | is less than 8, it will be displayed to standard output. now, append a |
193 | line to the foo.log file in the command line window, |
194 | |
195 | echo 'hello log' >> foo.log |
196 | |
197 | You will see in the archive hub window a log message appear in stdout |
198 | in 10 seconds (according to this configuration), |
199 | |
200 | 02/11/02 14:27:15 [tail][5] hello log |
201 | |
202 | This log entry is in raw format, the other Stem::Log configurations add |
203 | formats (as mentioned above). For more information on the format of the |
204 | log entries and filters please take a look at the log design notes. |
205 | |
206 | The final cell configuration is for filtering the status messages from the |
207 | LogTail Cell, |
208 | |
209 | [ |
210 | 'class' => 'Stem::Log', |
211 | 'args' => [ |
212 | |
213 | 'name' => 'bar_status', |
214 | 'path' => 'tail/bar_status.log', |
215 | 'format' => '[%f]%h:%H:%P - %T', |
216 | 'strftime' => '%T', |
217 | 'filters' => [ |
218 | file => 1, |
219 | 'env_gt_level' => 'bar_status', |
220 | tty_msg => 1, |
221 | ], |
222 | ], |
223 | ], |
224 | |
225 | As you can see, it has the same format as the previous Stem::Log |
226 | configurations. This configuration has both a logfile (tail/bar_status.log) |
227 | and the ability to display the status message if the severity level is less |
228 | than the Stem environment variable 'bar_status' to the tty message |
229 | console (indicated by the tty_msg filter operation, which is set to true). |
230 | Note that tty_msg is different than stdout, the message is being sent to |
231 | the TtyMsg module for output to the console versus just using stdout |
232 | directly. There are also other actions including custom ones. |
233 | |
234 | Let's see this in action, close the three widows created from the tail_demo |
235 | script and re-run tail_demo. do the following at the |
236 | Stem prompt in the archive hub window, |
237 | |
238 | bar_status=8 |
239 | |
240 | This will ensure that if the severity level of the incoming log entry |
241 | is less than 8, it will be displayed to standard output. now, append a |
242 | line to the foo.log file in the command line window, |
243 | |
244 | echo 'hello again log' >> foo.log |
245 | |
246 | You will see in the archive hub window a log message appear in stdout |
247 | in 10 seconds (according to this configuration), |
248 | |
249 | [21:58:04]trichards-linux.alias.net:monitor:/opt/bin/run_stem - LogTail: first open of /opt/bin/tail/foo.log |
250 | |
251 | |