1 package DBIx::Class::Storage::DBI::Replicated;
5 use Moose::Util::TypeConstraints;
6 use DBIx::Class::Storage::DBI;
7 use DBIx::Class::Storage::DBI::Replicated::Pool;
8 use DBIx::Class::Storage::DBI::Replicated::Balancer;
12 DBIx::Class::Storage::DBI::Replicated - ALPHA Replicated database support
16 The Following example shows how to change an existing $schema to a replicated
17 storage type, add some replicated (readonly) databases, and perform reporting
20 ## Change storage_type in your schema class
21 $schema->storage_type( ['::DBI::Replicated', {balancer=>'::Random'}] );
23 ## Add some slaves. Basically this is an array of arrayrefs, where each
24 ## arrayref is database connect information
26 $schema->storage->connect_replicants(
27 [$dsn1, $user, $pass, \%opts],
28 [$dsn2, $user, $pass, \%opts],
29 [$dsn3, $user, $pass, \%opts],
32 ## Now, just use the $schema as normal
33 $schema->resultset('Source')->search({name=>'etc'});
35 ## You can force a given query to use a particular storage using the search
36 ### attribute 'force_pool'. For example:
38 my $RS = $schema->resultset('Source')->search(undef, {force_pool=>'master'});
40 ## Now $RS will force everything (both reads and writes) to use whatever was
41 ## setup as the master storage. 'master' is hardcoded to always point to the
42 ## Master, but you can also use any Replicant name. Please see:
43 ## L<DBIx::Class::Storage::Replicated::Pool> and the replicants attribute for
44 ## More. Also see transactions and L</execute_reliably> for alternative ways
45 ## to force read traffic to the master.
49 Warning: This class is marked BETA. This has been running a production
50 website using MySQL native replication as it's backend and we have some decent
51 test coverage but the code hasn't yet been stressed by a variety of databases.
52 Individual DB's may have quirks we are not aware of. Please use this in first
53 development and pass along your experiences/bug fixes.
55 This class implements replicated data store for DBI. Currently you can define
56 one master and numerous slave database connections. All write-type queries
57 (INSERT, UPDATE, DELETE and even LAST_INSERT_ID) are routed to master
58 database, all read-type queries (SELECTs) go to the slave database.
60 Basically, any method request that L<DBIx::Class::Storage::DBI> would normally
61 handle gets delegated to one of the two attributes: L</read_handler> or to
62 L</write_handler>. Additionally, some methods need to be distributed
63 to all existing storages. This way our storage class is a drop in replacement
64 for L<DBIx::Class::Storage::DBI>.
66 Read traffic is spread across the replicants (slaves) occuring to a user
67 selected algorithm. The default algorithm is random weighted.
71 The consistancy betweeen master and replicants is database specific. The Pool
72 gives you a method to validate it's replicants, removing and replacing them
73 when they fail/pass predefined criteria. Please make careful use of the ways
74 to force a query to run against Master when needed.
78 This class defines the following attributes.
82 The underlying L<DBIx::Class::Schema> object this storage is attaching
88 isa=>'DBIx::Class::Schema',
95 Contains the classname which will instantiate the L</pool> object. Defaults
96 to: L<DBIx::Class::Storage::DBI::Replicated::Pool>.
104 default=>'DBIx::Class::Storage::DBI::Replicated::Pool',
106 'create_pool' => 'new',
112 Contains a hashref of initialized information to pass to the Balancer object.
113 See L<DBIx::Class::Storage::Replicated::Pool> for available arguments.
128 The replication pool requires a balance class to provider the methods for
129 choose how to spread the query load across each replicant in the pool.
133 subtype 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
136 coerce 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
141 $type = 'DBIx::Class::Storage::DBI::Replicated::Balancer'.$type;
143 Class::MOP::load_class($type);
147 has 'balancer_type' => (
149 isa=>'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
152 default=> 'DBIx::Class::Storage::DBI::Replicated::Balancer::First',
154 'create_balancer' => 'new',
160 Contains a hashref of initialized information to pass to the Balancer object.
161 See L<DBIx::Class::Storage::Replicated::Balancer> for available arguments.
165 has 'balancer_args' => (
175 Is a <DBIx::Class::Storage::DBI::Replicated::Pool> or derived class. This is a
176 container class for one or more replicated databases.
182 isa=>'DBIx::Class::Storage::DBI::Replicated::Pool',
193 Is a <DBIx::Class::Storage::DBI::Replicated::Balancer> or derived class. This
194 is a class that takes a pool (<DBIx::Class::Storage::DBI::Replicated::Pool>)
200 isa=>'DBIx::Class::Storage::DBI::Replicated::Balancer',
202 handles=>[qw/auto_validate_every/],
207 The master defines the canonical state for a pool of connected databases. All
208 the replicants are expected to match this databases state. Thus, in a classic
209 Master / Slaves distributed system, all the slaves are expected to replicate
210 the Master's state as quick as possible. This is the only database in the
211 pool of databases that is allowed to handle write traffic.
217 isa=>'DBIx::Class::Storage::DBI',
221 =head1 ATTRIBUTES IMPLEMENTING THE DBIx::Storage::DBI INTERFACE
223 The following methods are delegated all the methods required for the
224 L<DBIx::Class::Storage::DBI> interface.
228 Defines an object that implements the read side of L<BIx::Class::Storage::DBI>.
232 has 'read_handler' => (
245 Defines an object that implements the write side of L<BIx::Class::Storage::DBI>.
249 has 'write_handler' => (
262 deployment_statements
288 This class defines the following methods.
292 L<DBIx::Class::Schema> when instantiating it's storage passed itself as the
293 first argument. So we need to massage the arguments a bit so that all the
294 bits get put into the correct places.
299 my ($class, $schema, $storage_type_args, @args) = @_;
310 Lazy builder for the L</master> attribute.
316 DBIx::Class::Storage::DBI->new($self->schema);
321 Lazy builder for the L</pool> attribute.
327 $self->create_pool(%{$self->pool_args});
330 =head2 _build_balancer
332 Lazy builder for the L</balancer> attribute. This takes a Pool object so that
333 the balancer knows which pool it's balancing.
337 sub _build_balancer {
339 $self->create_balancer(
341 master=>$self->master,
342 %{$self->balancer_args},
346 =head2 _build_write_handler
348 Lazy builder for the L</write_handler> attribute. The default is to set this to
353 sub _build_write_handler {
354 return shift->master;
357 =head2 _build_read_handler
359 Lazy builder for the L</read_handler> attribute. The default is to set this to
364 sub _build_read_handler {
365 return shift->balancer;
368 =head2 around: connect_replicants
370 All calls to connect_replicants needs to have an existing $schema tacked onto
371 top of the args, since L<DBIx::Storage::DBI> needs it.
375 around 'connect_replicants' => sub {
376 my ($method, $self, @args) = @_;
377 $self->$method($self->schema, @args);
382 Returns an array of of all the connected storage backends. The first element
383 in the returned array is the master, and the remainings are each of the
390 return grep {defined $_ && blessed $_} (
396 =head2 execute_reliably ($coderef, ?@args)
398 Given a coderef, saves the current state of the L</read_handler>, forces it to
399 use reliable storage (ie sets it to the master), executes a coderef and then
400 restores the original state.
406 $schema->resultset('User')->create({name=>$name});
407 my $user_rs = $schema->resultset('User')->find({name=>$name});
411 my $user_rs = $schema->storage->execute_reliably($reliably, 'John');
413 Use this when you must be certain of your database state, such as when you just
414 inserted something and need to get a resultset including it, etc.
420 sub execute_reliably {
421 my ($self, $coderef, @args) = @_;
423 unless( ref $coderef eq 'CODE') {
424 $self->throw_exception('Second argument must be a coderef');
427 ##Get copy of master storage
428 my $master = $self->master;
430 ##Get whatever the current read hander is
431 my $current = $self->read_handler;
433 ##Set the read handler to master
434 $self->read_handler($master);
436 ## do whatever the caller needs
438 my $want_array = wantarray;
442 @result = $coderef->(@args);
443 } elsif(defined $want_array) {
444 ($result[0]) = ($coderef->(@args));
450 ##Reset to the original state
451 $self->read_handler($current);
453 ##Exception testing has to come last, otherwise you might leave the
454 ##read_handler set to master.
457 $self->throw_exception("coderef returned an error: $@");
459 return $want_array ? @result : $result[0];
463 =head2 set_reliable_storage
465 Sets the current $schema to be 'reliable', that is all queries, both read and
466 write are sent to the master
470 sub set_reliable_storage {
472 my $schema = $self->schema;
473 my $write_handler = $self->schema->storage->write_handler;
475 $schema->storage->read_handler($write_handler);
478 =head2 set_balanced_storage
480 Sets the current $schema to be use the </balancer> for all reads, while all
481 writea are sent to the master only
485 sub set_balanced_storage {
487 my $schema = $self->schema;
488 my $write_handler = $self->schema->storage->balancer;
490 $schema->storage->read_handler($write_handler);
493 =head2 around: txn_do ($coderef)
495 Overload to the txn_do method, which is delegated to whatever the
496 L<write_handler> is set to. We overload this in order to wrap in inside a
497 L</execute_reliably> method.
501 around 'txn_do' => sub {
502 my($txn_do, $self, $coderef, @args) = @_;
503 $self->execute_reliably(sub {$self->$txn_do($coderef, @args)});
508 Check that the master and at least one of the replicants is connected.
515 $self->master->connected &&
516 $self->pool->connected_replicants;
519 =head2 ensure_connected
521 Make sure all the storages are connected.
525 sub ensure_connected {
527 foreach my $source ($self->all_storages) {
528 $source->ensure_connected(@_);
534 Set the limit_dialect for all existing storages
540 foreach my $source ($self->all_storages) {
541 $source->limit_dialect(@_);
547 Set the quote_char for all existing storages
553 foreach my $source ($self->all_storages) {
554 $source->quote_char(@_);
560 Set the name_sep for all existing storages
566 foreach my $source ($self->all_storages) {
567 $source->name_sep(@_);
573 Set the schema object for all existing storages
579 foreach my $source ($self->all_storages) {
580 $source->set_schema(@_);
586 set a debug flag across all storages
592 foreach my $source ($self->all_storages) {
599 set a debug object across all storages
605 foreach my $source ($self->all_storages) {
606 $source->debugobj(@_);
612 set a debugfh object across all storages
618 foreach my $source ($self->all_storages) {
619 $source->debugfh(@_);
625 set a debug callback across all storages
631 foreach my $source ($self->all_storages) {
632 $source->debugcb(@_);
638 disconnect everything
644 foreach my $source ($self->all_storages) {
645 $source->disconnect(@_);
651 Due to the fact that replicants can lag behind a master, you must take care to
652 make sure you use one of the methods to force read queries to a master should
653 you need realtime data integrity. For example, if you insert a row, and then
654 immediately re-read it from the database (say, by doing $row->discard_changes)
655 or you insert a row and then immediately build a query that expects that row
656 to be an item, you should force the master to handle reads. Otherwise, due to
657 the lag, there is no certainty your data will be in the expected state.
659 For data integrity, all transactions automatically use the master storage for
660 all read and write queries. Using a transaction is the preferred and recommended
661 method to force the master to handle all read queries.
663 Otherwise, you can force a single query to use the master with the 'force_pool'
666 my $row = $resultset->search(undef, {force_pool=>'master'})->find($pk);
668 This attribute will safely be ignore by non replicated storages, so you can use
669 the same code for both types of systems.
671 Lastly, you can use the L</execute_reliably> method, which works very much like
674 For debugging, you can turn replication on/off with the methods L</set_reliable_storage>
675 and L</set_balanced_storage>, however this operates at a global level and is not
676 suitable if you have a shared Schema object being used by multiple processes,
677 such as on a web application server. You can get around this limitation by
678 using the Schema clone method.
680 my $new_schema = $schema->clone;
681 $new_schema->set_reliable_storage;
683 ## $new_schema will use only the Master storage for all reads/writes while
684 ## the $schema object will use replicated storage.
688 John Napiorkowski <john.napiorkowski@takkle.com>
690 Based on code originated by:
692 Norbert Csongrádi <bert@cpan.org>
693 Peter Siklósi <einon@einon.hu>
697 You may distribute this code under the same terms as Perl itself.
701 __PACKAGE__->meta->make_immutable;