From: John Napiorkowski Date: Wed, 31 Dec 2014 16:36:02 +0000 (-0600) Subject: merged and resolved conflicts from stable X-Git-Tag: 5.90079_008~21 X-Git-Url: http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=catagits%2FCatalyst-Runtime.git;a=commitdiff_plain;h=6dcc530761473f574ccde956e3a321b1dfb3d27e;hp=-c merged and resolved conflicts from stable --- 6dcc530761473f574ccde956e3a321b1dfb3d27e diff --combined Changes index 5014bfc,928442f..857ab36 --- a/Changes +++ b/Changes @@@ -1,63 -1,14 +1,75 @@@ # This file documents the revision history for Perl extension Catalyst. ++5.90079_005 - 2014-12-31 ++ - Merged changes from 5.90078 ++ +5.90079_004 - 2014-12-26 + - Starting adding some docs around the new encoding stuff + - Exposed the reqexp we use to match content types that need encoding via a + global variable. + - Added some test cases for JSON utf8 and tested file uploads with utf8. + - Fixes to decoding on file upload filenames and related methods + - new methods on upload object that tries to do the right thing if we find + a character set on the upload and its UTF8. + - new additional helper methods on the file upload object. + - new helper methods has_encoding and clear_encoding on context. + - Method on Catalyst::Response to determine if the reponse should be encoded. + - Warn if changing headers only if headers are finalized AND the response callback + has allready been called (and headers already sent). + - Centralized rules about detecting if we need to automatically encode or not and + added tests around cases when you choose to skip auto encoding. + +5.90079_003 - 2014-12-03 + - Make sure all tests run even if debug mode is enabled. + - Fixed issue with middleware stash test case that failed on older Perls + +5.90079_002 - 2014-12-02 + - Fixed typo in Makefile.PL which borked the previous distribution. No other + changes. + +5.90079_001 - 2014-12-02 + - MyApp->to_app is now an alias for MyApp->psgi_app in order to better support + existing Plack conventions. + - Modify Catayst::Response->from_psgi_response to allow the first argument to + be an object that does ->as_psgi. + - Modified Catayst::Middleware::Stash to be a shallow copy in $env. Added some + docs. Added a test case to make sure stash keys added in a child application + don't bubble back up to the main application. + - We no longer use Encode::is_utf8 since it doesn't work the way we think it + does... This required some UTF-8 changes. If your application is UTF-8 aware + I highly suggest you test this release. + - We alway do utf8 decoding on incoming URLs (before we only did so if the server + encoding was utf8. I believe this is correct as per the w3c spec, but please + correct if incorrect :) + - Debug output now shows utf8 characters if those are incoming via Args or as + path or pathparts in your actions. query and body parameter keys are now also + subject to utf8 decoding (or as specificed via the encoding configuration value). + - lots of UTF8 changes. Again we think this is now more correct but please test. + - Allow $c->res->redirect($url) to accept $url as an object that does ->as_string + which I think will ease a common case (and common bug) and added documentation. + - !!! UTF-8 is now the default encoding (there used to be none...). You can disable + this if you need to with MyApp->config(encoding => undef) if it causes you trouble. + - Calling $c->res->write($data) now encodes $data based on the configured encoding + (UTF-8 is default). + - $c->res->writer_fh now returns Catalyst::Response::Writer which is a decorator + over the PSGI writer and provides an additional methd 'write_encoded' that just + does the right thing for encoding your responses. This is probably the method + you want to use. + - New dispatch matching attribute: Scheme. This lets you match a route based on + the incoming URI scheme (http, https, ws, wss). + - If $c->uri_for targets an action or action chain that defines Scheme, use that + scheme for the generated URI object instead of just using whatever the incoming + request uses. + + 5.90078 - 2014-12-30 + - POD corrections (sergey++) + - New configuration option to disable the HTTP Exception passthru feature + introduced in 5.90060. You can use this if that feature is causing you + trouble. (davewood++); + - Some additional helper methods for dealing with errors. + - More clear exception when $request->body_data tries to parse malformed POSTed + data. Added documentation and tests around this. + 5.90077 - 2014-11-18 - We store the PSGI $env in Catalyst::Engine for backcompat reasons. Changed this so that the storage is a weak reference, so that it goes out of scope diff --combined lib/Catalyst.pm index 3ef4bc8,9686617..582ebb3 --- a/lib/Catalyst.pm +++ b/lib/Catalyst.pm @@@ -50,7 -50,7 +50,7 @@@ use Plack::Middleware::RemoveRedundantB use Catalyst::Middleware::Stash; use Plack::Util; use Class::Load 'load_class'; -use Encode 2.21 (); +use Encode 2.21 'decode_utf8', 'encode_utf8'; BEGIN { require 5.008003; } @@@ -86,10 -86,8 +86,10 @@@ has response => lazy => 1, ); sub _build_response_constructor_args { - my $self = shift; - { _log => $self->log }; + return +{ + _log => $_[0]->log, + encoding => $_[0]->encoding, + }; } has namespace => (is => 'rw'); @@@ -129,8 -127,7 +129,8 @@@ __PACKAGE__->stats_class('Catalyst::Sta __PACKAGE__->_encode_check(Encode::FB_CROAK | Encode::LEAVE_SRC); # Remember to update this in Catalyst::Runtime as well! - our $VERSION = '5.90079_004'; -our $VERSION = '5.90078'; ++our $VERSION = '5.90079_005'; +$VERSION = eval $VERSION if $VERSION =~ /_/; # numify for warning-free dev releases sub import { my ( $class, @arguments ) = @_; @@@ -498,18 -495,6 +498,18 @@@ Catalyst) # stash is automatically passed to the view for use in a template $c->forward( 'MyApp::View::TT' ); +The stash hash is currently stored in the PSGI C<$env> and is managed by +L. Since it's part of the C<$env> items in +the stash can be accessed in sub applications mounted under your main +L application. For example if you delegate the response of an +action to another L application, that sub application will have +access to all the stash keys of the main one, and if can of course add +more keys of its own. However those new keys will not 'bubble' back up +to the main application. + +For more information the best thing to do is to review the test case: +t/middleware-stash.t in the distribution /t directory. + =cut sub stash { @@@ -533,6 -518,9 +533,9 @@@ Add a new error $c->error('Something bad happened'); + Calling this will always return an arrayref (if there are no errors it + will be an empty arrayref. + =cut sub error { @@@ -577,6 -565,29 +580,29 @@@ Returns true if you have error sub has_errors { scalar(@{shift->error}) ? 1:0 } + =head2 $c->last_error + + Returns the most recent error in the stack (the one most recently added...) + or nothing if there are no errors. + + =cut + + sub last_error { my ($err, @errs) = @{shift->error}; return $err } + + =head2 shift_errors + + shifts the most recently added error off the error stack and returns if. Returns + nothing if there are nomore errors. + + =cut + + sub shift_errors { + my ($self) = @_; + my ($err, @errors) = @{$self->error}; + $self->{error} = \@errors; + return $err; + } + sub _comp_search_prefixes { my $c = shift; return map $c->components->{ $_ }, $c->_comp_names_search_prefixes(@_); @@@ -1013,31 -1024,12 +1039,31 @@@ And later Your log class should implement the methods described in L. +=head2 has_encoding + +Returned True if there's a valid encoding + +=head2 clear_encoding + +Clears the encoding for the current context + =head2 encoding Sets or gets the application encoding. =cut +sub has_encoding { shift->encoding ? 1:0 } + +sub clear_encoding { + my $c = shift; + if(blessed $c) { + $c->encoding(undef); + } else { + $c->debug->error("You can't clear encoding on the application"); + } +} + sub encoding { my $c = shift; my $encoding; @@@ -1354,8 -1346,6 +1380,8 @@@ sub setup_finalize =head2 $c->uri_for( $action, \@captures?, @args?, \%query_values? ) +=head2 $c->uri_for( $action, [@captures, @args], \%query_values? ) + Constructs an absolute L object based on the application root, the provided path, and the additional arguments and query parameters provided. When used as a string, provides a textual URI. If you need more flexibility @@@ -1395,10 -1385,6 +1421,10 @@@ path, use C<< $c->uri_for_action >> ins # Path to a static resource $c->uri_for('/static/images/logo.png'); +In general the scheme of the generated URI object will follow the incoming request +however if your targeted action or action chain has the Scheme attribute it will +use that instead. + =cut sub uri_for { @@@ -1416,38 -1402,30 +1442,38 @@@ ( scalar @args && ref $args[$#args] eq 'HASH' ? pop @args : {} ); carp "uri_for called with undef argument" if grep { ! defined $_ } @args; + + my @encoded_args = (); foreach my $arg (@args) { - utf8::encode($arg) if utf8::is_utf8($arg); - $arg =~ s/([^$URI::uric])/$URI::Escape::escapes{$1}/go; + if(ref($arg)||'' eq 'ARRAY') { + push @encoded_args, [map { + my $encoded = encode_utf8 $_; + $encoded =~ s/([^$URI::uric])/$URI::Escape::escapes{$1}/go; + $encoded; + } @$arg]; + } else { + push @encoded_args, do { + my $encoded = encode_utf8 $arg; + $encoded =~ s/([^$URI::uric])/$URI::Escape::escapes{$1}/go; + $encoded; + } + } } + my $target_action = $path->$_isa('Catalyst::Action') ? $path : undef; if ( $path->$_isa('Catalyst::Action') ) { # action object - s|/|%2F|g for @args; + s|/|%2F|g for @encoded_args; my $captures = [ map { s|/|%2F|g; $_; } - ( scalar @args && ref $args[0] eq 'ARRAY' - ? @{ shift(@args) } + ( scalar @encoded_args && ref $encoded_args[0] eq 'ARRAY' + ? @{ shift(@encoded_args) } : ()) ]; - foreach my $capture (@$captures) { - utf8::encode($capture) if utf8::is_utf8($capture); - $capture =~ s/([^$URI::uric])/$URI::Escape::escapes{$1}/go; - } - my $action = $path; # ->uri_for( $action, \@captures_and_args, \%query_values? ) - if( !@args && $action->number_of_args ) { + if( !@encoded_args && $action->number_of_args ) { my $expanded_action = $c->dispatcher->expand_action( $action ); - my $num_captures = $expanded_action->number_of_captures; - unshift @args, splice @$captures, $num_captures; + unshift @encoded_args, splice @$captures, $num_captures; } $path = $c->dispatcher->uri_for_action($action, $captures); @@@ -1459,37 -1437,25 +1485,37 @@@ $path = '/' if $path eq ''; } - unshift(@args, $path); + unshift(@encoded_args, $path); unless (defined $path && $path =~ s!^/!!) { # in-place strip my $namespace = $c->namespace; if (defined $path) { # cheesy hack to handle path '../foo' - $namespace =~ s{(?:^|/)[^/]+$}{} while $args[0] =~ s{^\.\./}{}; + $namespace =~ s{(?:^|/)[^/]+$}{} while $encoded_args[0] =~ s{^\.\./}{}; } - unshift(@args, $namespace || ''); + unshift(@encoded_args, $namespace || ''); } # join args with '/', or a blank string - my $args = join('/', grep { defined($_) } @args); + my $args = join('/', grep { defined($_) } @encoded_args); $args =~ s/\?/%3F/g; # STUPID STUPID SPECIAL CASE $args =~ s!^/+!!; my ($base, $class) = ('/', 'URI::_generic'); if(blessed($c)) { $base = $c->req->base; - $class = ref($base); + if($target_action) { + $target_action = $c->dispatcher->expand_action($target_action); + if(my $s = $target_action->scheme) { + $s = lc($s); + $class = "URI::$s"; + $base->scheme($s); + } else { + $class = ref($base); + } + } else { + $class = ref($base); + } + $base =~ s{(?{$_}; - s/([;\/?:@&=+,\$\[\]%])/$URI::Escape::escapes{$1}/go; + #s/([;\/?:@&=+,\$\[\]%])/$URI::Escape::escapes{$1}/go; ## Commented out because seems to lead to double encoding - JNAP s/ /+/g; my $key = $_; $val = '' unless defined $val; (map { my $param = "$_"; - utf8::encode( $param ) if utf8::is_utf8($param); + $param = encode_utf8($param); # using the URI::Escape pattern here so utf8 chars survive $param =~ s/([^A-Za-z0-9\-_.!~*'() ])/$URI::Escape::escapes{$1}/go; $param =~ s/ /+/g; + + $key = encode_utf8($key); + # using the URI::Escape pattern here so utf8 chars survive + $key =~ s/([^A-Za-z0-9\-_.!~*'() ])/$URI::Escape::escapes{$1}/go; + $key =~ s/ /+/g; + "${key}=$param"; } ( ref $val eq 'ARRAY' ? @$val : $val )); } @keys); } @@@ -1841,15 -1801,7 +1867,7 @@@ sub execute if ( my $error = $@ ) { #rethow if this can be handled by middleware - if( - blessed $error && ( - $error->can('as_psgi') || - ( - $error->can('code') && - $error->code =~m/^[1-5][0-9][0-9]$/ - ) - ) - ) { + if ( $c->_handle_http_exception($error) ) { foreach my $err (@{$c->error}) { $c->log->error($err); } @@@ -1962,7 -1914,7 +1980,7 @@@ sub finalize # Support skipping finalize for psgix.io style 'jailbreak'. Used to support # stuff like cometd and websockets - + if($c->request->_has_io_fh) { $c->log_response; return; @@@ -2030,10 -1982,7 +2048,7 @@@ sub finalize_error $c->engine->finalize_error( $c, @_ ); } else { my ($error) = @{$c->error}; - if( - blessed $error && - ($error->can('as_psgi') || $error->can('code')) - ) { + if ( $c->_handle_http_exception($error) ) { # In the case where the error 'knows what it wants', becauses its PSGI # aware, just rethow and let middleware catch it $error->can('rethrow') ? $error->rethrow : croak $error; @@@ -2068,8 -2017,6 +2083,8 @@@ sub finalize_headers $c->finalize_cookies; + # This currently is a NOOP but I don't want to remove it since I guess people + # might have Response subclasses that use it for something... (JNAP) $c->response->finalize_headers(); # Done @@@ -2078,49 -2025,42 +2093,49 @@@ =head2 $c->finalize_encoding -Make sure your headers and body are encoded properly IF you set an encoding. +Make sure your body is encoded properly IF you set an encoding. By +default the encoding is UTF-8 but you can disable it by explictly setting the +encoding configuration value to undef. + +We can only encode when the body is a scalar. Methods for encoding via the +streaming interfaces (such as C and C on L +are available). + See L. =cut sub finalize_encoding { my $c = shift; - - my $body = $c->response->body; - - return unless defined($body); - - my $enc = $c->encoding; - - return unless $enc; - - my ($ct, $ct_enc) = $c->response->content_type; - - # Only touch 'text-like' contents - return unless $c->response->content_type =~ /^text|xml$|javascript$/; - - if ($ct_enc && $ct_enc =~ /charset=([^;]*)/) { - if (uc($1) ne uc($enc->mime_name)) { - $c->log->debug("Unicode::Encoding is set to encode in '" . - $enc->mime_name . - "', content type is '$1', not encoding "); - return; - } - } else { - $c->res->content_type($c->res->content_type . "; charset=" . $enc->mime_name); + my $res = $c->res || return; + + # Warn if the set charset is different from the one you put into encoding. We need + # to do this early since encodable_response is false for this condition and we need + # to match the debug output for backcompat (there's a test for this...) -JNAP + if( + $res->content_type_charset and $c->encoding and + (uc($c->encoding->mime_name) ne uc($res->content_type_charset)) + ) { + my $ct = lc($res->content_type_charset); + $c->log->debug("Catalyst encoding config is set to encode in '" . + $c->encoding->mime_name . + "', content type is '$ct', not encoding "); } - # Oh my, I wonder what filehandle responses and streams do... - jnap. - # Encode expects plain scalars (IV, NV or PV) and segfaults on ref's - $c->response->body( $c->encoding->encode( $body, $c->_encode_check ) ) - if ref(\$body) eq 'SCALAR'; + if( + ($res->encodable_response) and + (defined($res->body)) and + (ref(\$res->body) eq 'SCALAR') + ) { + $c->res->body( $c->encoding->encode( $c->res->body, $c->_encode_check ) ); + + # Set the charset if necessary. This might be a bit bonkers since encodable response + # is false when the set charset is not the same as the encoding mimetype (maybe + # confusing action at a distance here.. + # Don't try to set the charset if one already exists + $c->res->content_type($c->res->content_type . "; charset=" . $c->encoding->mime_name) + unless($c->res->content_type_charset); + } } =head2 $c->finalize_output @@@ -2184,15 -2124,7 +2199,7 @@@ sub handle_request $status = $c->finalize; } catch { #rethow if this can be handled by middleware - if( - blessed($_) && ( - $_->can('as_psgi') || - ( - $_->can('code') && - $_->code =~m/^[1-5][0-9][0-9]$/ - ) - ) - ) { + if ( $class->_handle_http_exception($_) ) { $_->can('rethrow') ? $_->rethrow : croak $_; } chomp(my $error = $_); @@@ -2342,7 -2274,7 +2349,7 @@@ Prepares body parameters sub prepare_body_parameters { my $c = shift; - $c->engine->prepare_body_parameters( $c, @_ ); + $c->request->prepare_body_parameters( $c, @_ ); } =head2 $c->prepare_connection @@@ -2436,10 -2368,6 +2443,10 @@@ sub log_request $method ||= ''; $path = '/' unless length $path; $address ||= ''; + + $path =~ s/%([0-9A-Fa-f]{2})/chr(hex($1))/eg; + $path = decode_utf8($path); + $c->log->debug(qq/"$method" request for "$path" from "$address"/); $c->log_request_headers($request->headers); @@@ -2625,6 -2553,37 +2632,6 @@@ Prepares uploads sub prepare_uploads { my $c = shift; $c->engine->prepare_uploads( $c, @_ ); - - my $enc = $c->encoding; - return unless $enc; - - # Uggg we hook prepare uploads to do the encoding crap on post and query - # parameters! Sorry -jnap - for my $key (qw/ parameters query_parameters body_parameters /) { - for my $value ( values %{ $c->request->{$key} } ) { - # N.B. Check if already a character string and if so do not try to double decode. - # http://www.mail-archive.com/catalyst@lists.scsys.co.uk/msg02350.html - # this avoids exception if we have already decoded content, and is _not_ the - # same as not encoding on output which is bad news (as it does the wrong thing - # for latin1 chars for example).. - $value = $c->_handle_unicode_decoding($value); - } - } - for my $value ( values %{ $c->request->uploads } ) { - # skip if it fails for uploads, as we don't usually want uploads touched - # in any way - for my $inner_value ( ref($value) eq 'ARRAY' ? @{$value} : $value ) { - $inner_value->{filename} = try { - $enc->decode( $inner_value->{filename}, $c->_encode_check ) - } catch { - $c->handle_unicode_encoding_exception({ - param_value => $inner_value->{filename}, - error_msg => $_, - encoding_step => 'uploads', - }); - }; - } - } } =head2 $c->prepare_write @@@ -3057,9 -3016,7 +3064,9 @@@ sub apply_default_middlewares return $psgi_app; } -=head2 $c->psgi_app +=head2 App->psgi_app + +=head2 App->to_app Returns a PSGI application code reference for the catalyst application C<$c>. This is the bare application without any middlewares @@@ -3070,8 -3027,6 +3077,8 @@@ reference of your Catalyst application =cut +*to_app = \&psgi_app; + sub psgi_app { my ($app) = @_; my $psgi = $app->engine->build_psgi_app($app); @@@ -3102,20 -3057,14 +3109,20 @@@ sub setup_home =head2 $c->setup_encoding - Sets up the input/output encoding. See L + Sets up the input/output encoding. See L =cut sub setup_encoding { my $c = shift; - my $enc = delete $c->config->{encoding}; - $c->encoding( $enc ) if defined $enc; + if( exists($c->config->{encoding}) && !defined($c->config->{encoding}) ) { + # Ok, so the user has explicitly said "I don't want encoding..." + return; + } else { + my $enc = defined($c->config->{encoding}) ? + delete $c->config->{encoding} : 'UTF-8'; # not sure why we delete it... (JNAP) + $c->encoding($enc); + } } =head2 handle_unicode_encoding_exception @@@ -3153,13 -3102,8 +3160,13 @@@ sub _handle_unicode_decoding return $value; } elsif ( ref $value eq 'HASH' ) { - foreach ( values %$value ) { - $_ = $self->_handle_unicode_decoding($_); + foreach (keys %$value) { + my $encoded_key = $self->_handle_param_unicode_decoding($_); + $value->{$encoded_key} = $self->_handle_unicode_decoding($value->{$_}); + + # If the key was encoded we now have two (the original and current so + # delete the original. + delete $value->{$_} if $_ ne $encoded_key; } return $value; } @@@ -3174,7 -3118,9 +3181,7 @@@ sub _handle_param_unicode_decoding my $enc = $self->encoding; return try { - Encode::is_utf8( $value ) ? - $value - : $enc->decode( $value, $self->_encode_check ); + $enc->decode( $value, $self->_encode_check ); } catch { $self->handle_unicode_encoding_exception({ @@@ -3344,7 -3290,7 +3351,7 @@@ the plugin name does not begin with C @roles ) if @roles; } - } + } =head2 registered_middlewares @@@ -3403,7 -3349,7 +3410,7 @@@ sub registered_middlewares sub setup_middleware { my $class = shift; - my @middleware_definitions = @_ ? + my @middleware_definitions = @_ ? reverse(@_) : reverse(@{$class->config->{'psgi_middleware'}||[]}); my @middleware = (); @@@ -3495,12 -3441,34 +3502,34 @@@ sub default_data_handlers ->can('build_cgi_struct')->($params); }, 'application/json' => sub { - Class::Load::load_first_existing_class('JSON::MaybeXS', 'JSON') - ->can('decode_json')->(do { local $/; $_->getline }); - }, + my ($fh, $req) = @_; + my $parser = Class::Load::load_first_existing_class('JSON::MaybeXS', 'JSON'); + my $slurped; + return eval { + local $/; + $slurped = $fh->getline; + $parser->can("decode_json")->($slurped); + } || Catalyst::Exception->throw(sprintf "Error Parsing POST '%s', Error: %s", (defined($slurped) ? $slurped : 'undef') ,$@); + }, }; } + sub _handle_http_exception { + my ( $self, $error ) = @_; + if ( + !$self->config->{always_catch_http_exceptions} + && blessed $error + && ( + $error->can('as_psgi') + || ( $error->can('code') + && $error->code =~ m/^[1-5][0-9][0-9]$/ ) + ) + ) + { + return 1; + } + } + =head2 $c->stack Returns an arrayref of the internal execution stack (actions that are @@@ -3567,6 -3535,13 +3596,13 @@@ There are a number of 'base' config var =item * + C - As of version 5.90060 Catalyst + rethrows errors conforming to the interface described by + L and lets the middleware deal with it. + Set true to get the deprecated behaviour and have Catalyst catch HTTP exceptions. + + =item * + C - The default model picked if you say C<< $c->model >>. See L<< /$c->model($name) >>. =item * @@@ -3622,7 -3597,7 +3658,7 @@@ to be shown in hit debug tables in the =item * C - Controls if the C or C environment - variable should be used for determining the request path. + variable should be used for determining the request path. Most web server environments pass the requested path to the application using environment variables, from which Catalyst has to reconstruct the request base (i.e. the top level path to / in the application, @@@ -3661,7 -3636,7 +3697,7 @@@ is having paths rewritten into it (e.g at other URIs than that which the app is 'normally' based at with C), the resolution of C<< $c->request->base >> will be incorrect. - =back + =back =item * @@@ -3671,9 -3646,6 +3707,9 @@@ C - See L - See L +This now defaults to 'UTF-8'. You my turn it off by setting this configuration +value to undef. + =item * C @@@ -3682,7 -3654,7 +3718,7 @@@ When there is an error in an action cha processing the remaining actions and then catch the error upon chain end. This can lead to running actions when the application is in an unexpected state. If you have this issue, setting this config value to true will promptly exit a - chain when there is an error raised in any action (thus terminating the chain + chain when there is an error raised in any action (thus terminating the chain early.) use like: @@@ -3728,7 -3700,7 +3764,7 @@@ your stack, such as in a model that an is caught by Catalyst and unless you either catch it yourself (via eval or something like L or by reviewing the L stack, it will eventually reach L and return either the debugging - error stack page, or the default error page. However, if your exception + error stack page, or the default error page. However, if your exception can be caught by L, L will instead rethrow it so that it can be handled by that middleware (which is part of the default middleware). For example this would allow @@@ -3738,7 -3710,7 +3774,7 @@@ sub throws_exception :Local { my ($self, $c) = @_; - http_throw(SeeOther => { location => + http_throw(SeeOther => { location => $c->uri_for($self->action_for('redirect')) }); } @@@ -4019,36 -3991,6 +4055,36 @@@ Please see L for more on middlewa On request, decodes all params from encoding into a sequence of logical characters. On response, encodes body into encoding. +By default encoding is now 'UTF-8'. You may turn it off by setting +the encoding configuration to undef. + +Encoding is automatically applied when the content-type is set to +a type that can be encoded. Currently we encode when the content type +matches the following regular expression: + + $content_type =~ /^text|xml$|javascript$/ + +Encoding is set on the application, but it is copied to the response object +so you can override encoding rules per request (See L +for more information). + +Be default we don't automatically encode 'application/json' since the most +popular JSON encoders (such as L which is the library that +L can make use of) will do the UTF8 encoding and decoding automatically. +Having it on in Catalyst could result in double encoding. + +If you are producing JSON response in an unconventional manner (such +as via a template or manual strings) you should perform the UTF8 encoding +manually as well such as to conform to the JSON specification. + +NOTE: We also examine the value of $c->response->content_encoding. If +you set this (like for example 'gzip', and manually gzipping the body) +we assume that you have done all the neccessary encoding yourself, since +we cannot encode the gzipped contents. If you use a plugin like +L we will be updating that plugin to work +with the new UTF8 encoding code, or you can use L +or (probably best) do your compression on a front end proxy. + =head2 Methods =over 4 @@@ -4162,6 -4104,8 +4198,8 @@@ Chisel Wright C + davewood: David Schmidt + David Kamholz Edkamholz@cpan.orgE David Naughton, C diff --combined lib/Catalyst/Middleware/Stash.pm index e99285c,bd02b9c..e31f2d6 --- a/lib/Catalyst/Middleware/Stash.pm +++ b/lib/Catalyst/Middleware/Stash.pm @@@ -9,12 -9,12 +9,12 @@@ use Carp 'croak' our @EXPORT_OK = qw(stash get_stash); -sub PSGI_KEY { 'Catalyst.Stash.v1' }; +sub PSGI_KEY () { 'Catalyst.Stash.v1' } sub get_stash { my $env = shift; - return $env->{&PSGI_KEY} || - _init_stash_in($env); + return $env->{+PSGI_KEY} || + croak "You requested a stash, but one does not exist."; } sub stash { @@@ -38,16 -38,19 +38,16 @@@ sub _create_stash }; } -sub _init_stash_in { - my ($env) = @_; - return $env->{&PSGI_KEY} ||= - _create_stash; -} - sub call { my ($self, $env) = @_; - _init_stash_in($env); - return $self->app->($env); + my $new_env = +{ %$env }; + my %stash = %{ ($env->{+PSGI_KEY} || sub {})->() || +{} }; + + $new_env->{+PSGI_KEY} = _create_stash( \%stash ); + return $self->app->($new_env); } - =head1 TITLE + =head1 NAME Catalyst::Middleware::Stash - The Catalyst stash - in middleware @@@ -60,15 -63,6 +60,15 @@@ alone distributio We store a coderef under the C which can be dereferenced with key values or nothing to access the underly hashref. +The stash middleware is designed so that you can 'nest' applications that +use it. If for example you have a L application that is called +by a controller under a parent L application, the child application +will inherit the full stash of the parent BUT any new keys added by the child +will NOT bubble back up to the parent. However, children of children will. + +For more information the current test case t/middleware-stash.t is the best +documentation. + =head1 SUBROUTINES This class defines the following subroutines. @@@ -110,7 -104,7 +110,7 @@@ clients. Stash key / value are stored ["I found $stashed in the stash!"]]; }; -If the stash does not yet exist, we initialize one and return that. +If the stash does not yet exist, an exception is thrown. =head1 METHODS diff --combined lib/Catalyst/Request.pm index 0fe34b0,671dd51..5e57305 --- a/lib/Catalyst/Request.pm +++ b/lib/Catalyst/Request.pm @@@ -10,7 -10,7 +10,8 @@@ use HTTP::Headers use Stream::Buffered; use Hash::MultiValue; use Scalar::Util; +use HTTP::Body; + use Catalyst::Exception; use Moose; use namespace::clean -except => 'meta'; @@@ -118,7 -118,11 +119,11 @@@ has body_data => sub _build_body_data { my ($self) = @_; - my $content_type = $self->content_type; + + # Not sure if these returns should not be exceptions... + my $content_type = $self->content_type || return; + return unless ($self->method eq 'POST' || $self->method eq 'PUT'); + my ($match) = grep { $content_type =~/$_/i } keys(%{$self->data_handlers}); @@@ -127,7 -131,7 +132,7 @@@ local $_ = $fh; return $self->data_handlers->{$match}->($fh, $self); } else { - return undef; + Catalyst::Exception->throw("$content_type is does not have an available data handler"); } } @@@ -312,7 -316,7 +317,7 @@@ sub prepare_body_chunk } sub prepare_body_parameters { - my ( $self ) = @_; + my ( $self, $c ) = @_; $self->prepare_body if ! $self->_has_body; @@@ -320,29 -324,9 +325,29 @@@ return $self->_use_hash_multivalue ? Hash::MultiValue->new : {}; } + my $params = $self->_body->param; + + # If we have an encoding configured (like UTF-8) in general we expect a client + # to POST with the encoding we fufilled the request in. Otherwise don't do any + # encoding (good change wide chars could be in HTML entity style llike the old + # days -JNAP + + # so, now that HTTP::Body prepared the body params, we gotta 'walk' the structure + # and do any needed decoding. + + # This only does something if the encoding is set via the encoding param. Remember + # this is assuming the client is not bad and responds with what you provided. In + # general you can just use utf8 and get away with it. + # + # I need to see if $c is here since this also doubles as a builder for the object :( + + if($c and $c->encoding) { + $params = $c->_handle_unicode_decoding($params); + } + return $self->_use_hash_multivalue ? - Hash::MultiValue->from_mixed($self->_body->param) : - $self->_body->param; + Hash::MultiValue->from_mixed($params) : + $params; } sub prepare_connection { @@@ -522,6 -506,13 +527,13 @@@ data of the type 'application/json' an method. You may define addition data_handlers via a global configuration setting. See L for more information. + If the POST is malformed in some way (such as undefined or not content that + matches the content-type) we raise a L with the error + text as the message. + + If the POSTed content type does not match an availabled data handler, this + will also raise an exception. + =head2 $req->body_parameters Returns a reference to a hash containing body (POST) parameters. Values can @@@ -947,7 -938,7 +959,7 @@@ sub mangle_params next unless defined $value; for ( ref $value eq 'ARRAY' ? @$value : $value ) { $_ = "$_"; - utf8::encode( $_ ) if utf8::is_utf8($_); + # utf8::encode($_); } }; diff --combined lib/Catalyst/Runtime.pm index 465ffb0,4cc1f98..416f0de --- a/lib/Catalyst/Runtime.pm +++ b/lib/Catalyst/Runtime.pm @@@ -7,8 -7,7 +7,8 @@@ BEGIN { require 5.008003; # Remember to update this in Catalyst as well! - our $VERSION = '5.90079_004'; -our $VERSION = '5.90078'; ++our $VERSION = '5.90079_005'; +$VERSION = eval $VERSION if $VERSION =~ /_/; # numify for warning-free dev releases =head1 NAME