From: Matt S Trout Date: Sat, 8 Sep 2012 15:26:02 +0000 (+0000) Subject: extra testing rationale X-Git-Tag: v1.004002~5 X-Git-Url: http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=p5sagit%2Fstrictures.git;a=commitdiff_plain;h=f9df7e2ea53a48f6d536f5605441eed847ca7497 extra testing rationale --- diff --git a/Changes b/Changes index dc382ed..3ee4407 100644 --- a/Changes +++ b/Changes @@ -1,3 +1,4 @@ + - add better rationale for the extra testing heuristic 1.004001 - 2012-07-12 - test-specific strictures now enabled during 'dzil test' 1.004000 - 2012-07-12 diff --git a/lib/strictures.pm b/lib/strictures.pm index 6b44c3d..7575253 100644 --- a/lib/strictures.pm +++ b/lib/strictures.pm @@ -171,6 +171,60 @@ This method does the setup work described above in L This method traps the strictures->VERSION(1) call produced by a use line with a version number on it and does the version check. +=head1 EXTRA TESTING RATIONALE + +Every so often, somebody complains that they're deploying via 'git pull' +and that they don't want strictures to enable itself in this case - and that +setting C to 0 isn't acceptable (additional ways to +disable extra testing would be welcome but the discussion never seems to get +that far). + +In order to allow us to skip a couple of stages and get straight to a +productive conversation, here's my current rationale for turning the +extra testing on via a heuristic: + +The extra testing is all stuff that only ever blows up at compile time; +this is intentional. So the oft raised concern that it's different code being +tested is only sort of the case - none of the modules involved affect the +final optree to my knowledge, so the author gets some additional compile +time crashes which he/she then fixes, and the rest of the testing is +completely valid for all environments. + +The point of the extra testing - especially 'no indirect' - is to catch +mistakes that newbie users won't even realise are mistakes without +help. For example, + + foo { ... }; + +where foo is an & prototyped sub that you forgot to import - this is +pernicious to track down since all -seems- fine until it gets called +and you get a crash. Worse still, you can fail to have imported it due +to a circular require, at which point you have a load order dependent +bug which I've seen before now -only- show up in production due to tiny +differences between the production and the development environment. I wrote +L to explain +this particular problem before L itself existed. + +As such, in my experience so far the strictures extra testing has +-avoided- production versus development differences, not caused them. + +Additionally, strictures' policy is very much "try and provide as much +protection as possible for newbies - who won't think about whether there's +an option to turn on or not" - so having only the environment variable +is not sufficient to achieve that (I get to explain that you need to add +'use strict' at least once a week on freenode #perl - newbies sometimes +completely skip steps because they don't understand that that step +is important). + +I make no claims that the heuristic is perfect - it's already been evolved +significantly over time, especially for 1.004 where we changed things to +ensure it only fires on files in your checkout (rather than strictures-using +modules you happened to have installed, which was just silly). However, I +hope the above clarifies why a heuristic approach is not only necessary but +desirable from a POV of providing new users with as much safety as possible, +and will allow any future discussion on the subject to focus on "how do we +minimise annoyance to people deploying from checkouts intentionally". + =head1 COMMUNITY AND SUPPORT =head2 IRC channel