3 Catalyst::Manual::Tutorial::Testing - Catalyst Tutorial - Chapter 8: Testing
8 This is B<Chapter 8 of 10> for the Catalyst tutorial.
10 L<Tutorial Overview|Catalyst::Manual::Tutorial>
16 L<Introduction|Catalyst::Manual::Tutorial::Intro>
20 L<Catalyst Basics|Catalyst::Manual::Tutorial::CatalystBasics>
24 L<More Catalyst Basics|Catalyst::Manual::Tutorial::MoreCatalystBasics>
28 L<Basic CRUD|Catalyst::Manual::Tutorial::BasicCRUD>
32 L<Authentication|Catalyst::Manual::Tutorial::Authentication>
36 L<Authorization|Catalyst::Manual::Tutorial::Authorization>
40 L<Debugging|Catalyst::Manual::Tutorial::Debugging>
48 L<Advanced CRUD|Catalyst::Manual::Tutorial::AdvancedCRUD>
52 L<Appendices|Catalyst::Manual::Tutorial::Appendices>
59 You may have noticed that the Catalyst Helper scripts automatically
60 create basic C<.t> test scripts under the C<t> directory. This
61 chapter of the tutorial briefly looks at how these tests can be used
62 to not only ensure that your application is working correctly at the
63 present time, but also provide automated regression testing as you
64 upgrade various pieces of your application over time.
66 You can check out the source code for this example from the Catalyst
67 Subversion repository as per the instructions in
68 L<Catalyst::Manual::Tutorial::Intro|Catalyst::Manual::Tutorial::Intro>.
71 =head1 RUNNING THE "CANNED" CATALYST TESTS
73 There are a variety of ways to run Catalyst and Perl tests (for example,
74 C<perl Makefile.PL> and C<make test>), but one of the easiest is with the
75 C<prove> command. For example, to run all of the tests in the C<t>
80 There will be a lot of output because we have the C<-Debug> flag
81 enabled in C<lib/MyApp.pm> (see the C<CATALYST_DEBUG=0> tip below for
82 a quick and easy way to reduce the clutter). Look for lines like this
85 # Failed test 'Request should succeed'
86 # in t/controller_Books.t at line 8.
87 # Looks like you failed 1 test of 3.
89 The redirection used by the Authentication plugins will cause several
90 failures in the default tests. You can fix this by making the following
93 1) Change the line in C<t/01app.t> that reads:
95 ok( request('/')->is_success, 'Request should succeed' );
99 ok( request('/login')->is_success, 'Request should succeed' );
101 2) Change the "C<request('/logout')-E<gt>is_success>" to
102 "C<request('/logout')-E<gt>is_redirect>" in C<t/controller_Logout.t>.
104 3) Change the "C<request('/books')-E<gt>is_success>" to
105 "C<request('/books')-E<gt>is_redirect>" in C<t/controller_Books.t>.
107 4) Add "C<use MyApp;>" to the top of C<t/view_TT.t>.
109 As you can see in the C<prove> command line above, the C<--lib> option
110 is used to set the location of the Catalyst C<lib> directory. With this
111 command, you will get all of the usual development server debug output,
112 something most people prefer to disable while running tests cases.
113 Although you can edit the C<lib/MyApp.pm> to comment out the C<-Debug>
114 plugin, it's generally easier to simply set the C<CATALYST_DEBUG=0>
115 environment variable. For example:
117 $ CATALYST_DEBUG=0 prove --lib lib t
119 B<Note:> Depending on the versions of various modules you have
120 installed, you might get some C<used only once> warnings -- you can
121 ignore these. If you want to elliminate the warnings, you can
122 edit C<Template::Base> to disable and then re-enable warnings
123 are the C</usr/lib/perl5/Template/Base.pm> line in C<sub new>.
124 You can locate where C<Template::Base> is located with the
125 following command (it's probably in a place similar to
126 C</usr/lib/perl5/Template/Base.pm>):
128 perldoc -l Template::Base
130 Edit the file and modify C<sub new> to match:
133 { no strict qw( refs );
136 $argnames = \@{"$class\::BASEARGS"} || [ ];
137 # Turn warnings back on
142 During the C<t/02pod> and C<t/03podcoverage> tests, you might notice the
143 C<all skipped: set TEST_POD to enable this test> warning message. To
144 execute the Pod-related tests, add C<TEST_POD=1> to the C<prove>
147 $ CATALYST_DEBUG=0 TEST_POD=1 prove --lib lib t
149 If you omitted the Pod comments from any of the methods that were
150 inserted, you might have to go back and fix them to get these tests to
153 Another useful option is the C<verbose> (C<-v>) option to C<prove>. It
154 prints the name of each test case as it is being run:
156 $ CATALYST_DEBUG=0 TEST_POD=1 prove --lib lib -v t
159 =head1 RUNNING A SINGLE TEST
161 You can also run a single script by appending its name to the C<prove>
162 command. For example:
164 $ CATALYST_DEBUG=0 prove --lib lib t/01app.t
166 Also note that you can also run tests directly from Perl without C<prove>.
169 $ CATALYST_DEBUG=0 perl -Ilib t/01app.t
172 =head1 ADDING YOUR OWN TEST SCRIPT
174 Although the Catalyst helper scripts provide a basic level of checks
175 "for free," testing can become significantly more helpful when you write
176 your own script to exercise the various parts of your application. The
177 L<Test::WWW::Mechanize::Catalyst|Test::WWW::Mechanize::Catalyst> module
178 is very popular for writing these sorts of test cases. This module
179 extends L<Test::WWW::Mechanize|Test::WWW::Mechanize> (and therefore
180 L<WWW::Mechanize|WWW::Mechanize>) to allow you to automate the action of
181 a user "clicking around" inside your application. It gives you all the
182 benefits of testing on a live system without the messiness of having to
183 use an actual web server, and a real person to do the clicking.
185 To create a sample test case, open the C<t/live_app01.t> file in your
186 editor and enter the following:
193 # Load testing framework and use 'no_plan' to dynamically pick up
194 # all tests. Better to replace "'no_plan'" with "tests => 30" so it
195 # knows exactly how many tests need to be run (and will tell you if
196 # not), but 'no_plan' is nice for quick & dirty tests
198 use Test::More 'no_plan';
200 # Need to specify the name of your app as arg on next line
202 # use Test::WWW::Mechanize::Catalyst "MyApp";
204 use ok "Test::WWW::Mechanize::Catalyst" => "MyApp";
206 # Create two 'user agents' to simulate two different users ('test01' & 'test02')
207 my $ua1 = Test::WWW::Mechanize::Catalyst->new;
208 my $ua2 = Test::WWW::Mechanize::Catalyst->new;
210 # Use a simplified for loop to do tests that are common to both users
211 # Use get_ok() to make sure we can hit the base URL
212 # Second arg = optional description of test (will be displayed for failed tests)
213 # Note that in test scripts you send everything to 'http://localhost'
214 $_->get_ok("http://localhost/", "Check redirect of base URL") for $ua1, $ua2;
215 # Use title_is() to check the contents of the <title>...</title> tags
216 $_->title_is("Login", "Check for login title") for $ua1, $ua2;
217 # Use content_contains() to match on text in the html body
218 $_->content_contains("You need to log in to use this application",
219 "Check we are NOT logged in") for $ua1, $ua2;
221 # Log in as each user
222 # Specify username and password on the URL
223 $ua1->get_ok("http://localhost/login?username=test01&password=mypass", "Login 'test01'");
224 # Use the form for user 'test02'; note there is no description here
227 username => 'test02',
228 password => 'mypass',
231 # Go back to the login page and it should show that we are already logged in
232 $_->get_ok("http://localhost/login", "Return to '/login'") for $ua1, $ua2;
233 $_->title_is("Login", "Check for login page") for $ua1, $ua2;
234 $_->content_contains("Please Note: You are already logged in as ",
235 "Check we ARE logged in" ) for $ua1, $ua2;
237 # 'Click' the 'Logout' link (see also 'text_regex' and 'url_regex' options)
238 $_->follow_link_ok({n => 4}, "Logout via first link on page") for $ua1, $ua2;
239 $_->title_is("Login", "Check for login title") for $ua1, $ua2;
240 $_->content_contains("You need to log in to use this application",
241 "Check we are NOT logged in") for $ua1, $ua2;
244 $ua1->get_ok("http://localhost/login?username=test01&password=mypass", "Login 'test01'");
245 $ua2->get_ok("http://localhost/login?username=test02&password=mypass", "Login 'test02'");
246 # Should be at the Book List page... do some checks to confirm
247 $_->title_is("Book List", "Check for book list title") for $ua1, $ua2;
249 $ua1->get_ok("http://localhost/books/list", "'test01' book list");
250 $ua1->get_ok("http://localhost/login", "Login Page");
251 $ua1->get_ok("http://localhost/books/list", "'test01' book list");
253 $_->content_contains("Book List", "Check for book list title") for $ua1, $ua2;
254 # Make sure the appropriate logout buttons are displayed
255 $_->content_contains("/logout\">User Logout</a>",
256 "Both users should have a 'User Logout'") for $ua1, $ua2;
257 $ua1->content_contains("/books/form_create\">Create</a>",
258 "Only 'test01' should have a create link");
260 $ua1->get_ok("http://localhost/books/list", "View book list as 'test01'");
262 # User 'test01' should be able to create a book with the "formless create" URL
263 $ua1->get_ok("http://localhost/books/url_create/TestTitle/2/4",
264 "'test01' formless create");
265 $ua1->title_is("Book Created", "Book created title");
266 $ua1->content_contains("Added book 'TestTitle'", "Check title added OK");
267 $ua1->content_contains("by 'Stevens'", "Check author added OK");
268 $ua1->content_contains("with a rating of 2.", "Check rating added");
269 # Try a regular expression to combine the previous 3 checks & account for whitespace
270 $ua1->content_like(qr/Added book 'TestTitle'\s+by 'Stevens'\s+with a rating of 2./, "Regex check");
272 # Make sure the new book shows in the list
273 $ua1->get_ok("http://localhost/books/list", "'test01' book list");
274 $ua1->title_is("Book List", "Check logged in and at book list");
275 $ua1->content_contains("Book List", "Book List page test");
276 $ua1->content_contains("TestTitle", "Look for 'TestTitle'");
278 # Make sure the new book can be deleted
279 # Get all the Delete links on the list page
280 my @delLinks = $ua1->find_all_links(text => 'Delete');
281 # Use the final link to delete the last book
282 $ua1->get_ok($delLinks[$#delLinks]->url, 'Delete last book');
283 # Check that delete worked
284 $ua1->content_contains("Book List", "Book List page test");
285 $ua1->content_contains("Book deleted", "Book was deleted");
287 # User 'test02' should not be able to add a book
288 $ua2->get_ok("http://localhost/books/url_create/TestTitle2/2/5", "'test02' add");
289 $ua2->content_contains("Unauthorized!", "Check 'test02' cannot add");
291 The C<live_app.t> test cases uses copious comments to explain each step
292 of the process. In addition to the techniques shown here, there are a
293 variety of other methods available in
294 L<Test::WWW::Mechanize::Catalyst|Test::WWW::Mechanize::Catalyst> (for
295 example, regex-based matching). Consult the documentation for more
298 B<TIP>: For I<unit tests> vs. the "full application tests" approach used
299 by L<Test::WWW::Mechanize::Catalyst|Test::WWW::Mechanize::Catalyst>, see
300 L<Catalyst::Test|Catalyst::Test>.
302 B<Note:> The test script does not test the C<form_create> and
303 C<form_create_do> actions. That is left as an exercise for the reader
304 (you should be able to complete that logic using the existing code as a
307 To run the new test script, use a command such as:
309 $ CATALYST_DEBUG=0 prove --lib lib -v t/live_app01.t
313 $ DBIC_TRACE=0 CATALYST_DEBUG=0 prove --lib lib -v t/live_app01.t
315 Experiment with the C<DBIC_TRACE>, C<CATALYST_DEBUG> and C<-v>
316 settings. If you find that there are errors, use the techniques
317 discussed in the "Catalyst Debugging" section (Chapter 7) to isolate
318 and fix any problems.
320 If you want to run the test case under the Perl interactive debugger,
321 try a command such as:
323 $ DBIC_TRACE=0 CATALYST_DEBUG=0 perl -d -Ilib t/live_app01.t
325 Note that although this tutorial uses a single custom test case for
326 simplicity, you may wish to break your tests into different files for
329 B<TIP:> If you have a test case that fails, you will receive an error
330 similar to the following:
332 # Failed test 'Check we are NOT logged in'
333 # in t/live_app01.t at line 31.
334 # searched: "\x{0a}<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Tran"...
335 # can't find: "You need to log in to use this application."
337 Unfortunately, this only shows us the first 50 characters of the HTML
338 returned by the request -- not enough to determine where the problem
339 lies. A simple technique that can be used in such situations is to
340 temporarily insert a line similar to the following right after the
345 This will cause the full HTML returned by the request to be displayed.
347 Another approach to see the full HTML content at the failure point in
348 a series of tests would be to insert a "C<$DB::single=1;> right above
349 the location of the failure and run the test under the perl debugger
350 (with C<-d>) as shown above. Then you can use the debugger to explore
351 the state of the application right before or after the failure.
354 =head1 SUPPORTING BOTH PRODUCTION AND TEST DATABASES
356 You may wish to leverage the techniques discussed in this tutorial to
357 maintain both a "production database" for your live application and a
358 "testing database" for your test cases. One advantage to
359 L<Test::WWW::Mechanize::Catalyst|Test::WWW::Mechanize::Catalyst> is that
360 it runs your full application; however, this can complicate things when
361 you want to support multiple databases. One solution is to allow the
362 database specification to be overridden with an environment variable.
363 For example, open C<lib/MyApp/Model/DB.pm> in your editor and
364 change the C<__PACKAGE__-E<gt>config(...> declaration to resemble:
366 my $dsn = $ENV{MYAPP_DSN} ||= 'dbi:SQLite:myapp.db';
368 schema_class => 'MyApp::Schema',
374 Then, when you run your test case, you can use commands such as:
376 $ cp myapp.db myappTEST.db
377 $ CATALYST_DEBUG=0 MYAPP_DSN="dbi:SQLite:myappTEST.db" prove --lib lib -v t/live_app01.t
379 This will modify the DSN only while the test case is running. If you
380 launch your normal application without the C<MYAPP_DSN> environment
381 variable defined, it will default to the same C<dbi:SQLite:myapp.db> as
387 Kennedy Clark, C<hkclark@gmail.com>
389 Please report any errors, issues or suggestions to the author. The
390 most recent version of the Catalyst Tutorial can be found at
391 L<http://dev.catalyst.perl.org/repos/Catalyst/Catalyst-Manual/5.70/trunk/lib/Catalyst/Manual/Tutorial/>.
393 Copyright 2006-2008, Kennedy Clark, under Creative Commons License
394 (L<http://creativecommons.org/licenses/by-sa/3.0/us/>).