cleaned up demo scripts locations
[urisagit/Stem.git] / demo / DEMO_TAIL
CommitLineData
4536f655 1
2 Demonstration of Log Tail
3
4This demonstration script illustrates Stem's ability to monitor log
5files. It showcases the Stem modules Stem::Log (which logs messages)
6and Stem::LogTail (which checks files for changes/updates). This
7demonstration runs two hubs named archive and monitor. The monitor
8hub watches a particular log file for changes. When a change occurs,
9messages are sent to the archive hub to be logged. The archive
10hub records the contents of the monitored log file (sent by the
11monitor hub) and also records status messages sent by the monitor
12hub. Log messages that are recorded by the archive hub can be
13stored as either raw data or with custom formats. This demonstrates
14a single log file being monitored, in a real world case there could be
15several log files being monitored. It is easy to see in this example that
16Stem can handle this with a small number of additions to its
17configuration files. This can be distributed securely across a network.
18This demo script is described in detail below with sections on
19running, using, configuring, and experimenting with it.
20
21Running tail_demo
22
23The log tail demonstration is called tail_demo and it uses monitor.stem
24and archive.stem configuration files (found in conf/). It is run with
25the simple command:
26
27tail_demo
28
29To exit, just enter 'q' in the tail_demo script window itself. It will
30kill all the other windows and then exit. This will also happen if you
31interrupt the demo script with ^C.
32
33If the -s option is used, then all the windows will use ssfe (split
34screen front end) which provides a command line edit and history window
35section and an output section. Two hub windows named Archive and
36Monitor will be created and a single shell window will be created with its
37current directory set to tail/. Stem will create two log files
38in the tail/ directory, bar.log and bar_status.log. bar.log is used
39by the archive hub to record what ever is sent to that log file and
40bar_status.log is used as a log file for status messages.
41The hub windows can be used to interact with that hubs Stem environment
42and the command line window can be used to put contents into a foo.log
43file. The two hub windows use the standard module Stem::TtyMsg which
44allows you to interact with them. In this demo they will be used to
45modify the Stem environment which will affect the behavior of the
46logical logs.
47
48Using tail_demo
49
50Initially, bar.log and bar_status.log will be empty files but in 10
51seconds (the tailing interval set in the monitor.stem configuration)
52the status log will have a message about foo.log not being found. Run
53ls -l several times in the shell (center) window to see when the status
54has been logged and then read that file will
55
56$ cat bar_status.log
57
58Now type the following at the command line (in the shell window):
59
60$ echo 'foobar' > foo.log
61
62After 10 seconds (configured in the Stem configuration file)
63you can look in bar.log and you will notice that there is a single line
64that reads, "foobar", and in bar_status.log you will notice that there
65is a status message saying that it is the first time that foo.log was
66opened. And, of course, we have the line "foobar" in the monitored log
67file, foo.log.
68
69Configuring tail_demo
70
71Look at the file conf/monitor.stem. That is one of the configuration files
72used by tail_demo. It is very simple and easy to understand. It is a Perl list
73of lists structure with key/value pairs. Read the config_notes for more
74on this.
75
76The first Cell configured is Stem::Hub which names this hub as
77'monitor'.
78
79 [
80 class => 'Stem::Hub',
81 name => 'monitor',
82 args => [],
83 ],
84
85Next comes the configuration for the Stem::Portal cell,
86
87 [
88 class => 'Stem::Portal',
89 args => [
90 ],
91 ],
92
93It is important to note here that there are no args passed into the
94portal. This means that the portal is a client portal, its default
95host is set to localhost, and its default port is set to 10,000. For
96more information on portals, read the portal design notes.
97
98The next Cell configured is Stem::TtyMsg which supports typing in and
99sending command messages. This is used in all the demo configurations.
100
101 [
102 class => 'Stem::TtyMsg',
103 args => [],
104 ],
105
106The next cell is the application specific part in the monitor.stem
107configuration, the Cell configuration for Stem::LogTail:
108
109 [
110 'class' => 'Stem::LogTail',
111 'name' => 'foo',
112 'args' => [
113 'path' => 'tail/foo.log',
114 'repeat_interval' => 10,
115 'data_log' => 'archive:bar',
116 'status_log' => 'archive:bar_status',
117 ],
118 ],
119
120This is the cell responsible for monitoring the indicated log file
121(foo.log). It has arguments for the path to the monitored file, what
122the time interval is (in seconds) to check the file for changes, the
123data log, and the status log. Note that the address of the data and status
124log indicates both the name of the hub that it is located at, as well as, the
125name of the log cell. For more information on these configuration options
126please see the tail log design notes.
127
128Now, lets take a look at the conf/archive.stem configuration file. This
129file defines logical log files (bar and bar_status) that are used by the
130monitor.stem configuration file to log the changes to foo.log.
131
132The first three cells that are configured are the same as the monitor
133configuration, Stem::Hub, Stem::Portal, and Stem::TtyMsg. They are
134for the most part identical. It is worth mentioning here that the
135configuration for the Portal has its server boolean flag set to true,
136indicating that this portal will be awaiting connection requests from
137remote Portals anywhere on a network.
138
139The next three are the configurations for Stem::Log logical logs. The
140first one is a typical logical log file configuration,
141
142 [
143 'class' => 'Stem::Log',
144 'args' => [
145 'name' => 'bar',
146 'path' => 'tail/bar.log',
147 'filters' => [
148 file => 1,
149 forward => [ 'bar_stdout' ],
150 ],
151 ],
152 ],
153
154This is defining a logical log named "bar" that is associated with
155a real log file indicated by the path, "tail/bar.log". It also
156has a filters argument that allows Stem::Log::Entries to be filtered
157before they are placed in the log file. The first of the filter
158operations in the above configuration, 'file', indicates that the
159incoming log entry should be placed in the file indicated by the 'path'
160argument. Another one of these filter rules, 'forward, indicates that
161the log entry is always forwarded to the log 'bar_stdout'.
162
163
164The next Stem::Log configuration defines a logical log that conditionally
165outputs its entries to STDOUT. It will write log file entries to STDOUT
166if the 'bar_stdout' Stem environment variable is set to be greater than
167the severity level of the log entry.
168
169 [
170 'class' => 'Stem::Log',
171 'args' => [
172
173 'name' => 'bar_stdout',
174 'format' => '%f [%L][%l] %T',
175 'strftime' => '%D %T',
176 'filters' => [
177 'env_gt_level' => 'bar_stdout',
178 stdout => 1,
179 ],
180 ],
181 ],
182
183This configuration specifies a format (overriding the default raw format
184like bar.log) that displays the entry with timestamps, label, and level.
185This allows you to customize your log entries to your liking.
186To demonstrate the severity level detection, do the following at the
187Stem prompt in the archive hub window (upper left),
188
189bar_stdout=8
190
191This will ensure that if the severity level of the incoming log entry
192is less than 8, it will be displayed to standard output. now, append a
193line to the foo.log file in the command line window,
194
195echo 'hello log' >> foo.log
196
197You will see in the archive hub window a log message appear in stdout
198in 10 seconds (according to this configuration),
199
20002/11/02 14:27:15 [tail][5] hello log
201
202This log entry is in raw format, the other Stem::Log configurations add
203formats (as mentioned above). For more information on the format of the
204log entries and filters please take a look at the log design notes.
205
206The final cell configuration is for filtering the status messages from the
207LogTail Cell,
208
209 [
210 'class' => 'Stem::Log',
211 'args' => [
212
213 'name' => 'bar_status',
214 'path' => 'tail/bar_status.log',
215 'format' => '[%f]%h:%H:%P - %T',
216 'strftime' => '%T',
217 'filters' => [
218 file => 1,
219 'env_gt_level' => 'bar_status',
220 tty_msg => 1,
221 ],
222 ],
223 ],
224
225As you can see, it has the same format as the previous Stem::Log
226configurations. This configuration has both a logfile (tail/bar_status.log)
227and the ability to display the status message if the severity level is less
228than the Stem environment variable 'bar_status' to the tty message
229console (indicated by the tty_msg filter operation, which is set to true).
230Note that tty_msg is different than stdout, the message is being sent to
231the TtyMsg module for output to the console versus just using stdout
232directly. There are also other actions including custom ones.
233
234Let's see this in action, close the three widows created from the tail_demo
235script and re-run tail_demo. do the following at the
236Stem prompt in the archive hub window,
237
238bar_status=8
239
240This will ensure that if the severity level of the incoming log entry
241is less than 8, it will be displayed to standard output. now, append a
242line to the foo.log file in the command line window,
243
244echo 'hello again log' >> foo.log
245
246You will see in the archive hub window a log message appear in stdout
247in 10 seconds (according to this configuration),
248
249[21:58:04]trichards-linux.alias.net:monitor:/opt/bin/run_stem - LogTail: first open of /opt/bin/tail/foo.log
250
251