=head1 DESCRIPTION
-This module is a subclass of L<SQL::Abstract> and includes a number of
-DBIC-specific workarounds, not yet suitable for inclusion into the
+This module is currently a subclass of L<SQL::Abstract> and includes a number of
+DBIC-specific extensions/workarounds, not suitable for inclusion into the
L<SQL::Abstract> core. It also provides all (and more than) the functionality
of L<SQL::Abstract::Limit>, see L<DBIx::Class::SQLMaker::LimitDialects> for
more info.
-Currently the enhancements to L<SQL::Abstract> are:
+Currently the enhancements over L<SQL::Abstract> are:
=over
=item * C<GROUP BY>/C<HAVING> support (via extensions to the order_by parameter)
+=item * A rudimentary multicolumn IN operator
+
=item * Support of C<...FOR UPDATE> type of select statement modifiers
-=item * The L</-ident> operator
+=back
+
+=head1 ROADMAP
+
+Some maintainer musings on the current state of SQL generation within DBIC as
+of Oct 2015
+
+=head2 Folding of most (or all) of L<SQL::Abstract (SQLA)|SQL::Abstract> into DBIC
+
+The rise of complex prefetch use, and the general streamlining of result
+parsing within DBIC ended up pushing the actual SQL generation to the forefront
+of many casual performance profiles. While the idea behind SQLA's API is sound,
+the actual implementation is terribly inefficient (once again bumping into the
+ridiculously high overhead of perl function calls).
+
+Given that SQLA has a B<very> distinct life on its own, and is used within an
+order of magnitude more projects compared to DBIC, it is prudent to B<not>
+disturb the current call chains within SQLA itself. Instead in the near future
+an effort will be undertaken to seek a more thorough decoupling of DBIC SQL
+generation from reliance on SQLA, possibly to a point where B<DBIC will no
+longer depend on SQLA> at all.
+
+B<The L<SQL::Abstract> library itself will continue being maintained> although
+it is not likely to gain many extra features, notably dialect support, at least
+not within the base C<SQL::Abstract> namespace.
-=item * The L</-value> operator
+This work (if undertaken) will take into consideration the following
+constraints:
+
+=over
+
+=item Main API compatibility
+
+The object returned by C<< $schema->storage->sqlmaker >> needs to be able to
+satisfy most of the basic tests found in the current-at-the-time SQLA dist.
+While things like L<case|SQL::Abstract/case> or L<logic|SQL::Abstract/logic>
+or even worse L<convert|SQL::Abstract/convert> will definitely remain
+unsupported, the rest of the tests should pass (within reason).
+
+=item Ability to plug back an SQL::Abstract (or derivative)
+
+During the initial work on L<Data::Query> the test suite of DBIC turned out to
+be an invaluable asset to iron out hard-to-reason-about corner cases. In
+addition the test suite is much more vast and intricate than the tests of SQLA
+itself. This state of affairs is way too valuable to sacrifice in order to gain
+faster SQL generation. Thus a compile-time-ENV-check will be introduced along
+with an extra CI configuration to ensure that DBIC is used with an off-the-CPAN
+SQLA and that it continues to flawlessly run its entire test suite. While this
+will undoubtedly complicate the implementation of the better performing SQL
+generator, it will preserve both the usability of the test suite for external
+projects and will keep L<SQL::Abstract> from regressions in the future.
=back
+Aside from these constraints it is becoming more and more practical to simply
+stop using SQLA in day-to-day production deployments of DBIC. The flexibility
+of the internals is simply not worth the performance cost.
+
+=head2 Relationship to L<Data::Query (DQ)|Data::Query>
+
+When initial work on DQ was taking place, the tools in L<::Storage::DBIHacks
+|http://github.com/dbsrgits/dbix-class/blob/master/lib/DBIx/Class/Storage/DBIHacks.pm>
+were only beginning to take shape, and it wasn't clear how important they will
+become further down the road. In fact the I<regexing all over the place> was
+considered an ugly stop-gap, and even a couple of highly entertaining talks
+were given to that effect. As the use-cases of DBIC were progressing, and
+evidence for the importance of supporting arbitrary SQL was mounting, it became
+clearer that DBIC itself would not really benefit in any way from an
+integration with DQ, but on the contrary is likely to lose functionality while
+the corners of the brand new DQ codebase are sanded off.
+
+The current status of DBIC/DQ integration is that the only benefit is for DQ by
+having access to the very extensive "early adopter" test suite, in the same
+manner as early DBIC benefitted tremendously from usurping the Class::DBI test
+suite. As far as the DBIC user-base - there are no immediate practical upsides
+to DQ integration, neither in terms of API nor in performance.
+
+So (as described higher up) the DBIC development effort will in the foreseable
+future ignore the existence of DQ, and will continue optimizing the preexisting
+SQLA-based solution, potentially "organically growing" its own compatible
+implementation. Also (again, as described higher up) the ability to plug a
+separate SQLA-compatible class providing the necessary surface API will remain
+possible, and will be protected at all costs in order to continue providing DQ
+access to the test cases of DBIC.
+
+In the short term, after one more pass over the ResultSet internals is
+undertaken I<real soon now (tm)>, and before the SQLA/SQLMaker integration
+takes place, the preexisting DQ-based branches will be pulled/modified/rebased
+to get up-to-date with the current state of the codebase, which changed very
+substantially since the last migration effort, especially in the SQL
+classification meta-parsing codepath.
+
=cut
use base qw/
/;
use mro 'c3';
-use Sub::Name 'subname';
use DBIx::Class::Carp;
-use DBIx::Class::Exception;
+use DBIx::Class::_Util 'set_subname';
+use SQL::Abstract 'is_literal_value';
use namespace::clean;
__PACKAGE__->mk_group_accessors (simple => qw/quote_char name_sep limit_dialect/);
+sub _quoting_enabled {
+ ( defined $_[0]->{quote_char} and length $_[0]->{quote_char} ) ? 1 : 0
+}
+
# for when I need a normalized l/r pair
sub _quote_chars {
+
+ # in case we are called in the old !!$sm->_quote_chars fashion
+ return () if !wantarray and ( ! defined $_[0]->{quote_char} or ! length $_[0]->{quote_char} );
+
map
{ defined $_ ? $_ : '' }
( ref $_[0]->{quote_char} ? (@{$_[0]->{quote_char}}) : ( ($_[0]->{quote_char}) x 2 ) )
# that use DBIx::Class::Carp/DBIx::Class::Exception instead of plain Carp
no warnings qw/redefine/;
- *SQL::Abstract::belch = subname 'SQL::Abstract::belch' => sub (@) {
+ *SQL::Abstract::belch = set_subname 'SQL::Abstract::belch' => sub (@) {
my($func) = (caller(1))[3];
carp "[$func] Warning: ", @_;
};
- *SQL::Abstract::puke = subname 'SQL::Abstract::puke' => sub (@) {
+ *SQL::Abstract::puke = set_subname 'SQL::Abstract::puke' => sub (@) {
my($func) = (caller(1))[3];
__PACKAGE__->throw_exception("[$func] Fatal: " . join ('', @_));
};
-
- # Current SQLA pollutes its namespace - clean for the time being
- namespace::clean->clean_subroutines(qw/SQL::Abstract carp croak confess/);
}
# the "oh noes offset/top without limit" constant
# as the value to abuse with MSSQL ordered subqueries)
sub __max_int () { 0x7FFFFFFF };
+# we ne longer need to check this - DBIC has ways of dealing with it
+# specifically ::Storage::DBI::_resolve_bindattrs()
+sub _assert_bindval_matches_bindtype () { 1 };
+
# poor man's de-qualifier
sub _quote {
- $_[0]->next::method( ( $_[0]{_dequalify_idents} and ! ref $_[1] )
+ $_[0]->next::method( ( $_[0]{_dequalify_idents} and defined $_[1] and ! ref $_[1] )
? $_[1] =~ / ([^\.]+) $ /x
: $_[1]
);
}
-sub new {
- my $self = shift->next::method(@_);
-
- # use the same coderefs, they are prepared to handle both cases
- my @extra_dbic_syntax = (
- { regex => qr/^ ident $/xi, handler => '_where_op_IDENT' },
- { regex => qr/^ value $/xi, handler => '_where_op_VALUE' },
- );
-
- push @{$self->{special_ops}}, @extra_dbic_syntax;
- push @{$self->{unary_ops}}, @extra_dbic_syntax;
-
- $self;
-}
-
-sub _where_op_IDENT {
- my $self = shift;
- my ($op, $rhs) = splice @_, -2;
- if (ref $rhs) {
- $self->throw_exception("-$op takes a single scalar argument (a quotable identifier)");
- }
-
- # in case we are called as a top level special op (no '=')
- my $lhs = shift;
-
- $_ = $self->_convert($self->_quote($_)) for ($lhs, $rhs);
-
- return $lhs
- ? "$lhs = $rhs"
- : $rhs
- ;
-}
-
-sub _where_op_VALUE {
- my $self = shift;
- my ($op, $rhs) = splice @_, -2;
-
- # in case we are called as a top level special op (no '=')
- my $lhs = shift;
-
- my @bind = [
- ($lhs || $self->{_nested_func_lhs} || $self->throw_exception("Unable to find bindtype for -value $rhs") ),
- $rhs
- ];
-
- return $lhs
- ? (
- $self->_convert($self->_quote($lhs)) . ' = ' . $self->_convert('?'),
- @bind
- )
- : (
- $self->_convert('?'),
- @bind,
- )
- ;
-}
-
sub _where_op_NEST {
carp_unique ("-nest in search conditions is deprecated, you most probably wanted:\n"
.q|{..., -and => [ \%cond0, \@cond1, \'cond2', \[ 'cond3', [ col => bind ] ], etc. ], ... }|
sub select {
my ($self, $table, $fields, $where, $rs_attrs, $limit, $offset) = @_;
+ ($fields, @{$self->{select_bind}}) = length ref $fields
+ ? $self->_recurse_fields( $fields )
+ : $self->_quote( $fields )
+ ;
- $fields = $self->_recurse_fields($fields);
+ # Override the default behavior of SQL::Abstract - SELECT * makes
+ # no sense in the context of DBIC (and has resulted in several
+ # tricky debugging sessions in the past)
+ not length $fields
+ and
+# FIXME - some day we need to enable this, but too many things break
+# ( notably S::L )
+# # Random value selected by a fair roll of dice
+# # In seriousness - this has to be a number, as it is much more
+# # palatable to random engines in a SELECT list
+# $fields = 42
+# and
+ carp_unique (
+ "ResultSets with an empty selection are deprecated (you almost certainly "
+ . "did not mean to do that): if this is indeed your intent you must "
+ . "explicitly supply \\'*' to your search()"
+ );
if (defined $offset) {
$self->throw_exception('A supplied offset must be a non-negative integer')
- if ( $offset =~ /\D/ or $offset < 0 );
+ if ( $offset =~ /[^0-9]/ or $offset < 0 );
}
$offset ||= 0;
if (defined $limit) {
$self->throw_exception('A supplied limit must be a positive integer')
- if ( $limit =~ /\D/ or $limit <= 0 );
+ if ( $limit =~ /[^0-9]/ or $limit <= 0 );
}
elsif ($offset) {
$limit = $self->__max_int;
($sql, @bind) = $self->next::method ($table, $fields, $where);
- my $limiter =
- $self->can ('emulate_limit') # also backcompat hook from SQLA::Limit
- ||
- do {
- my $dialect = $self->limit_dialect
- or $self->throw_exception( "Unable to generate SQL-limit - no limit dialect specified on $self, and no emulate_limit method found" );
- $self->can ("_$dialect")
- or $self->throw_exception(__PACKAGE__ . " does not implement the requested dialect '$dialect'");
- }
- ;
+ my $limiter;
+
+ if( $limiter = $self->can ('emulate_limit') ) {
+ carp_unique(
+ 'Support for the legacy emulate_limit() mechanism inherited from '
+ . 'SQL::Abstract::Limit has been deprecated, and will be removed at '
+ . 'some future point, as it gets in the way of architectural and/or '
+ . 'performance advances within DBIC. If your code uses this type of '
+ . 'limit specification please file an RT and provide the source of '
+ . 'your emulate_limit() implementation, so an acceptable upgrade-path '
+ . 'can be devised'
+ );
+ }
+ else {
+ my $dialect = $self->limit_dialect
+ or $self->throw_exception( "Unable to generate SQL-limit - no limit dialect specified on $self" );
+
+ $limiter = $self->can ("_$dialect")
+ or $self->throw_exception(__PACKAGE__ . " does not implement the requested dialect '$dialect'");
+ }
$sql = $self->$limiter (
$sql,
};
sub _lock_select {
my ($self, $type) = @_;
- my $sql = $for_syntax->{$type} || $self->throw_exception( "Unknown SELECT .. FOR type '$type' requested" );
+
+ my $sql;
+ if (ref($type) eq 'SCALAR') {
+ $sql = "FOR $$type";
+ }
+ else {
+ $sql = $for_syntax->{$type} || $self->throw_exception( "Unknown SELECT .. FOR type '$type' requested" );
+ }
+
return " $sql";
}
# optimized due to hotttnesss
# my ($self, $table, $data, $options) = @_;
- # SQLA will emit INSERT INTO $table ( ) VALUES ( )
+ # FIXME SQLA will emit INSERT INTO $table ( ) VALUES ( )
# which is sadly understood only by MySQL. Change default behavior here,
- # until SQLA2 comes with proper dialect support
+ # until we fold the extra pieces into SQLMaker properly
if (! $_[2] or (ref $_[2] eq 'HASH' and !keys %{$_[2]} ) ) {
my @bind;
my $sql = sprintf(
sub _recurse_fields {
my ($self, $fields) = @_;
- my $ref = ref $fields;
- return $self->_quote($fields) unless $ref;
- return $$fields if $ref eq 'SCALAR';
- if ($ref eq 'ARRAY') {
- return join(', ', map { $self->_recurse_fields($_) } @$fields);
+ if( not length ref $fields ) {
+ return $self->_quote( $fields );
+ }
+
+ elsif( my $lit = is_literal_value( $fields ) ) {
+ return @$lit
+ }
+
+ elsif( ref $fields eq 'ARRAY' ) {
+ my (@select, @bind, @bind_fragment);
+
+ (
+ ( $select[ $#select + 1 ], @bind_fragment ) = length ref $_
+ ? $self->_recurse_fields( $_ )
+ : $self->_quote( $_ )
+ ),
+ ( push @bind, @bind_fragment )
+ for @$fields;
+
+ return (join(', ', @select), @bind);
}
- elsif ($ref eq 'HASH') {
+
+ # FIXME - really crappy handling of functions
+ elsif ( ref $fields eq 'HASH') {
my %hash = %$fields; # shallow copy
my $as = delete $hash{-as}; # if supplied
- my ($func, $args, @toomany) = %hash;
+ my ($func, $rhs, @toomany) = %hash;
# there should be only one pair
- if (@toomany) {
- $self->throw_exception( "Malformed select argument - too many keys in hash: " . join (',', keys %$fields ) );
- }
+ $self->throw_exception(
+ "Malformed select argument - too many keys in hash: " . join (',', keys %$fields )
+ ) if @toomany;
+
+ $self->throw_exception (
+ 'The select => { distinct => ... } syntax is not supported for multiple columns.'
+ .' Instead please use { group_by => [ qw/' . (join ' ', @$rhs) . '/ ] }'
+ .' or { select => [ qw/' . (join ' ', @$rhs) . '/ ], distinct => 1 }'
+ ) if (
+ lc ($func) eq 'distinct'
+ and
+ ref $rhs eq 'ARRAY'
+ and
+ @$rhs > 1
+ );
- if (lc ($func) eq 'distinct' && ref $args eq 'ARRAY' && @$args > 1) {
- $self->throw_exception (
- 'The select => { distinct => ... } syntax is not supported for multiple columns.'
- .' Instead please use { group_by => [ qw/' . (join ' ', @$args) . '/ ] }'
- .' or { select => [ qw/' . (join ' ', @$args) . '/ ], distinct => 1 }'
- );
- }
+ my ($rhs_sql, @rhs_bind) = length ref $rhs
+ ? $self->_recurse_fields($rhs)
+ : $self->_quote($rhs)
+ ;
- my $select = sprintf ('%s( %s )%s',
- $self->_sqlcase($func),
- $self->_recurse_fields($args),
- $as
- ? sprintf (' %s %s', $self->_sqlcase('as'), $self->_quote ($as) )
- : ''
+ return(
+ sprintf( '%s( %s )%s',
+ $self->_sqlcase($func),
+ $rhs_sql,
+ $as
+ ? sprintf (' %s %s', $self->_sqlcase('as'), $self->_quote ($as) )
+ : ''
+ ),
+ @rhs_bind
);
-
- return $select;
- }
- # Is the second check absolutely necessary?
- elsif ( $ref eq 'REF' and ref($$fields) eq 'ARRAY' ) {
- push @{$self->{select_bind}}, @{$$fields}[1..$#$$fields];
- return $$fields->[0];
}
+
else {
- $self->throw_exception( $ref . qq{ unexpected in _recurse_fields()} );
+ $self->throw_exception( ref($fields) . ' unexpected in _recurse_fields()' );
}
}
# What we have been doing forever is hijacking the $order arg of
# SQLA::select to pass in arbitrary pieces of data (first the group_by,
# then pretty much the entire resultset attr-hash, as more and more
-# things in the SQLA space need to have mopre info about the $rs they
+# things in the SQLA space need to have more info about the $rs they
# create SQL for. The alternative would be to keep expanding the
# signature of _select with more and more positional parameters, which
-# is just gross. All hail SQLA2!
+# is just gross.
+#
+# FIXME - this will have to transition out to a subclass when the effort
+# of folding the SQLA machinery into SQLMaker takes place
sub _parse_rs_attrs {
my ($self, $arg) = @_;
my $sql = '';
+ my @sqlbind;
- if ($arg->{group_by}) {
- # horible horrible, waiting for refactor
- local $self->{select_bind};
- if (my $g = $self->_recurse_fields($arg->{group_by}) ) {
- $sql .= $self->_sqlcase(' group by ') . $g;
- push @{$self->{group_bind} ||= []}, @{$self->{select_bind}||[]};
- }
+ if (
+ $arg->{group_by}
+ and
+ @sqlbind = $self->_recurse_fields($arg->{group_by})
+ ) {
+ $sql .= $self->_sqlcase(' group by ') . shift @sqlbind;
+ push @{$self->{group_bind}}, @sqlbind;
}
- if (defined $arg->{having}) {
- my ($frag, @bind) = $self->_recurse_where($arg->{having});
- push(@{$self->{having_bind}}, @bind);
- $sql .= $self->_sqlcase(' having ') . $frag;
+ if (
+ $arg->{having}
+ and
+ @sqlbind = $self->_recurse_where($arg->{having})
+ ) {
+ $sql .= $self->_sqlcase(' having ') . shift @sqlbind;
+ push(@{$self->{having_bind}}, @sqlbind);
}
- if (defined $arg->{order_by}) {
+ if ($arg->{order_by}) {
+ # unlike the 2 above, _order_by injects into @{...bind...} for us
$sql .= $self->_order_by ($arg->{order_by});
}
my ($self, $arg) = @_;
# check that we are not called in legacy mode (order_by as 4th argument)
- if (ref $arg eq 'HASH' and not grep { $_ =~ /^-(?:desc|asc)/i } keys %$arg ) {
- return $self->_parse_rs_attrs ($arg);
- }
- else {
- my ($sql, @bind) = $self->next::method($arg);
- push @{$self->{order_bind}}, @bind;
- return $sql;
- }
+ (
+ ref $arg eq 'HASH'
+ and
+ not grep { $_ =~ /^-(?:desc|asc)/i } keys %$arg
+ )
+ ? $self->_parse_rs_attrs ($arg)
+ : do {
+ my ($sql, @bind) = $self->next::method($arg);
+ push @{$self->{order_bind}}, @bind;
+ $sql; # RV
+ }
+ ;
+}
+
+sub _split_order_chunk {
+ my ($self, $chunk) = @_;
+
+ # strip off sort modifiers, but always succeed, so $1 gets reset
+ $chunk =~ s/ (?: \s+ (ASC|DESC) )? \s* $//ix;
+
+ return (
+ $chunk,
+ ( $1 and uc($1) eq 'DESC' ) ? 1 : 0,
+ );
}
sub _table {
sub _recurse_from {
my $self = shift;
-
return join (' ', $self->_gen_from_blocks(@_) );
}
# Backcompat for the old days when a plain hashref
# { 't1.col1' => 't2.col2' } meant ON t1.col1 = t2.col2
- # Once things settle we should start warning here so that
- # folks unroll their hacks
if (
ref $cond eq 'HASH'
and
and
! ref ( (values %$cond)[0] )
) {
+ carp_unique(
+ "ResultSet {from} structures with conditions not conforming to the "
+ . "SQL::Abstract syntax are deprecated: you either need to stop abusing "
+ . "{from} altogether, or express the condition properly using the "
+ . "{ -ident => ... } operator"
+ );
$cond = { keys %$cond => { -ident => values %$cond } }
}
elsif ( ref $cond eq 'ARRAY' ) {
return $self->_recurse_where($cond);
}
-1;
-
-=head1 OPERATORS
-
-=head2 -ident
-
-Used to explicitly specify an SQL identifier. Takes a plain string as value
-which is then invariably treated as a column name (and is being properly
-quoted if quoting has been requested). Most useful for comparison of two
-columns:
-
- my %where = (
- priority => { '<', 2 },
- requestor => { -ident => 'submitter' }
- );
-
-which results in:
-
- $stmt = 'WHERE "priority" < ? AND "requestor" = "submitter"';
- @bind = ('2');
-
-=head2 -value
-
-The -value operator signals that the argument to the right is a raw bind value.
-It will be passed straight to DBI, without invoking any of the SQL::Abstract
-condition-parsing logic. This allows you to, for example, pass an array as a
-column value for databases that support array datatypes, e.g.:
+# !!! EXPERIMENTAL API !!! WILL CHANGE !!!
+#
+# This is rather odd, but vanilla SQLA does not have support for multicolumn IN
+# expressions
+# Currently has only one callsite in ResultSet, body moved into this subclass
+# of SQLA to raise API questions like:
+# - how do we convey a list of idents...?
+# - can binds reside on lhs?
+#
+# !!! EXPERIMENTAL API !!! WILL CHANGE !!!
+sub _where_op_multicolumn_in {
+ my ($self, $lhs, $rhs) = @_;
+
+ if (! ref $lhs or ref $lhs eq 'ARRAY') {
+ my (@sql, @bind);
+ for (ref $lhs ? @$lhs : $lhs) {
+ if (! ref $_) {
+ push @sql, $self->_quote($_);
+ }
+ elsif (ref $_ eq 'SCALAR') {
+ push @sql, $$_;
+ }
+ elsif (ref $_ eq 'REF' and ref $$_ eq 'ARRAY') {
+ my ($s, @b) = @$$_;
+ push @sql, $s;
+ push @bind, @b;
+ }
+ else {
+ $self->throw_exception("ARRAY of @{[ ref $_ ]}es unsupported for multicolumn IN lhs...");
+ }
+ }
+ $lhs = \[ join(', ', @sql), @bind];
+ }
+ elsif (ref $lhs eq 'SCALAR') {
+ $lhs = \[ $$lhs ];
+ }
+ elsif (ref $lhs eq 'REF' and ref $$lhs eq 'ARRAY' ) {
+ # noop
+ }
+ else {
+ $self->throw_exception( ref($lhs) . "es unsupported for multicolumn IN lhs...");
+ }
- my %where = (
- array => { -value => [1, 2, 3] }
- );
+ # is this proper...?
+ $rhs = \[ $self->_recurse_where($rhs) ];
-which results in:
+ for ($lhs, $rhs) {
+ $$_->[0] = "( $$_->[0] )"
+ unless $$_->[0] =~ /^ \s* \( .* \) \s* $/xs;
+ }
- $stmt = 'WHERE array = ?';
- @bind = ([1, 2, 3]);
+ \[ join( ' IN ', shift @$$lhs, shift @$$rhs ), @$$lhs, @$$rhs ];
+}
-=head1 AUTHORS
+=head1 FURTHER QUESTIONS?
-See L<DBIx::Class/CONTRIBUTORS>.
+Check the list of L<additional DBIC resources|DBIx::Class/GETTING HELP/SUPPORT>.
-=head1 LICENSE
+=head1 COPYRIGHT AND LICENSE
-You may distribute this code under the same terms as Perl itself.
+This module is free software L<copyright|DBIx::Class/COPYRIGHT AND LICENSE>
+by the L<DBIx::Class (DBIC) authors|DBIx::Class/AUTHORS>. You can
+redistribute it and/or modify it under the same terms as the
+L<DBIx::Class library|DBIx::Class/COPYRIGHT AND LICENSE>.
=cut
+
+1;
projects / dbsrgits/DBIx-Class.git / blobdiff