1 package DBIx::Class::Storage::DBI::Replicated;
5 use Moose::Util::TypeConstraints;
6 use DBIx::Class::Storage::DBI;
7 use DBIx::Class::Storage::DBI::Replicated::Pool;
8 use DBIx::Class::Storage::DBI::Replicated::Balancer;
12 DBIx::Class::Storage::DBI::Replicated - ALPHA Replicated database support
16 The Following example shows how to change an existing $schema to a replicated
17 storage type, add some replicated (readonly) databases, and perform reporting
20 ## Change storage_type in your schema class
21 $schema->storage_type( ['::DBI::Replicated', {balancer=>'::Random'}] );
23 ## Add some slaves. Basically this is an array of arrayrefs, where each
24 ## arrayref is database connect information
26 $schema->storage->connect_replicants(
27 [$dsn1, $user, $pass, \%opts],
28 [$dsn2, $user, $pass, \%opts],
29 [$dsn3, $user, $pass, \%opts],
32 ## Now, just use the $schema as normal
33 $schema->resultset('Source')->search({name=>'etc'});
35 ## You can force a given query to use a particular storage using the search
36 ### attribute 'force_pool'. For example:
38 my $RS = $schema->resultset('Source')->search(undef, {force_pool=>'master'});
40 ## Now $RS will force everything (both reads and writes) to use whatever was
41 ## setup as the master storage. 'master' is hardcoded to always point to the
42 ## Master, but you can also use any Replicant name. Please see:
43 ## L<DBIx::Class::Storage::Replicated::Pool> and the replicants attribute for
44 ## More. Also see transactions and L</execute_reliably> for alternative ways
45 ## to force read traffic to the master.
49 Warning: This class is marked BETA. This has been running a production
50 website using MySQL native replication as it's backend and we have some decent
51 test coverage but the code hasn't yet been stressed by a variety of databases.
52 Individual DB's may have quirks we are not aware of. Please use this in first
53 development and pass along your experiences/bug fixes.
55 This class implements replicated data store for DBI. Currently you can define
56 one master and numerous slave database connections. All write-type queries
57 (INSERT, UPDATE, DELETE and even LAST_INSERT_ID) are routed to master
58 database, all read-type queries (SELECTs) go to the slave database.
60 Basically, any method request that L<DBIx::Class::Storage::DBI> would normally
61 handle gets delegated to one of the two attributes: L</read_handler> or to
62 L</write_handler>. Additionally, some methods need to be distributed
63 to all existing storages. This way our storage class is a drop in replacement
64 for L<DBIx::Class::Storage::DBI>.
66 Read traffic is spread across the replicants (slaves) occuring to a user
67 selected algorithm. The default algorithm is random weighted.
71 The consistancy betweeen master and replicants is database specific. The Pool
72 gives you a method to validate it's replicants, removing and replacing them
73 when they fail/pass predefined criteria. Please make careful use of the ways
74 to force a query to run against Master when needed.
78 This class defines the following attributes.
82 The underlying L<DBIx::Class::Schema> object this storage is attaching
88 isa=>'DBIx::Class::Schema',
95 Contains the classname which will instantiate the L</pool> object. Defaults
96 to: L<DBIx::Class::Storage::DBI::Replicated::Pool>.
104 default=>'DBIx::Class::Storage::DBI::Replicated::Pool',
106 'create_pool' => 'new',
112 Contains a hashref of initialized information to pass to the Balancer object.
113 See L<DBIx::Class::Storage::Replicated::Pool> for available arguments.
128 The replication pool requires a balance class to provider the methods for
129 choose how to spread the query load across each replicant in the pool.
133 subtype 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
136 coerce 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
141 $type = 'DBIx::Class::Storage::DBI::Replicated::Balancer'.$type;
143 Class::MOP::load_class($type);
147 has 'balancer_type' => (
149 isa=>'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
152 default=> 'DBIx::Class::Storage::DBI::Replicated::Balancer::First',
154 'create_balancer' => 'new',
160 Contains a hashref of initialized information to pass to the Balancer object.
161 See L<DBIx::Class::Storage::Replicated::Balancer> for available arguments.
165 has 'balancer_args' => (
175 Is a <DBIx::Class::Storage::DBI::Replicated::Pool> or derived class. This is a
176 container class for one or more replicated databases.
182 isa=>'DBIx::Class::Storage::DBI::Replicated::Pool',
193 Is a <DBIx::Class::Storage::DBI::Replicated::Balancer> or derived class. This
194 is a class that takes a pool (<DBIx::Class::Storage::DBI::Replicated::Pool>)
200 isa=>'DBIx::Class::Storage::DBI::Replicated::Balancer',
202 handles=>[qw/auto_validate_every/],
207 The master defines the canonical state for a pool of connected databases. All
208 the replicants are expected to match this databases state. Thus, in a classic
209 Master / Slaves distributed system, all the slaves are expected to replicate
210 the Master's state as quick as possible. This is the only database in the
211 pool of databases that is allowed to handle write traffic.
217 isa=>'DBIx::Class::Storage::DBI',
221 =head1 ATTRIBUTES IMPLEMENTING THE DBIx::Storage::DBI INTERFACE
223 The following methods are delegated all the methods required for the
224 L<DBIx::Class::Storage::DBI> interface.
228 Defines an object that implements the read side of L<BIx::Class::Storage::DBI>.
232 has 'read_handler' => (
245 Defines an object that implements the write side of L<BIx::Class::Storage::DBI>.
249 has 'write_handler' => (
262 deployment_statements
288 This class defines the following methods.
292 L<DBIx::Class::Schema> when instantiating it's storage passed itself as the
293 first argument. So we need to massage the arguments a bit so that all the
294 bits get put into the correct places.
298 around 'new' => sub {
299 my ($new, $self, $schema, $storage_type_args, @args) = @_;
300 return $self->$new(schema=>$schema, %$storage_type_args, @args);
305 Lazy builder for the L</master> attribute.
311 DBIx::Class::Storage::DBI->new($self->schema);
316 Lazy builder for the L</pool> attribute.
322 $self->create_pool(%{$self->pool_args});
325 =head2 _build_balancer
327 Lazy builder for the L</balancer> attribute. This takes a Pool object so that
328 the balancer knows which pool it's balancing.
332 sub _build_balancer {
334 $self->create_balancer(
336 master=>$self->master,
337 %{$self->balancer_args},
341 =head2 _build_write_handler
343 Lazy builder for the L</write_handler> attribute. The default is to set this to
348 sub _build_write_handler {
349 return shift->master;
352 =head2 _build_read_handler
354 Lazy builder for the L</read_handler> attribute. The default is to set this to
359 sub _build_read_handler {
360 return shift->balancer;
363 =head2 around: connect_replicants
365 All calls to connect_replicants needs to have an existing $schema tacked onto
366 top of the args, since L<DBIx::Storage::DBI> needs it.
370 around 'connect_replicants' => sub {
371 my ($method, $self, @args) = @_;
372 $self->$method($self->schema, @args);
377 Returns an array of of all the connected storage backends. The first element
378 in the returned array is the master, and the remainings are each of the
385 return grep {defined $_ && blessed $_} (
391 =head2 execute_reliably ($coderef, ?@args)
393 Given a coderef, saves the current state of the L</read_handler>, forces it to
394 use reliable storage (ie sets it to the master), executes a coderef and then
395 restores the original state.
401 $schema->resultset('User')->create({name=>$name});
402 my $user_rs = $schema->resultset('User')->find({name=>$name});
406 my $user_rs = $schema->storage->execute_reliably($reliably, 'John');
408 Use this when you must be certain of your database state, such as when you just
409 inserted something and need to get a resultset including it, etc.
413 sub execute_reliably {
414 my ($self, $coderef, @args) = @_;
416 unless( ref $coderef eq 'CODE') {
417 $self->throw_exception('Second argument must be a coderef');
420 ##Get copy of master storage
421 my $master = $self->master;
423 ##Get whatever the current read hander is
424 my $current = $self->read_handler;
426 ##Set the read handler to master
427 $self->read_handler($master);
429 ## do whatever the caller needs
431 my $want_array = wantarray;
435 @result = $coderef->(@args);
436 } elsif(defined $want_array) {
437 ($result[0]) = ($coderef->(@args));
443 ##Reset to the original state
444 $self->read_handler($current);
446 ##Exception testing has to come last, otherwise you might leave the
447 ##read_handler set to master.
450 $self->throw_exception("coderef returned an error: $@");
452 return $want_array ? @result : $result[0];
456 =head2 set_reliable_storage
458 Sets the current $schema to be 'reliable', that is all queries, both read and
459 write are sent to the master
463 sub set_reliable_storage {
465 my $schema = $self->schema;
466 my $write_handler = $self->schema->storage->write_handler;
468 $schema->storage->read_handler($write_handler);
471 =head2 set_balanced_storage
473 Sets the current $schema to be use the </balancer> for all reads, while all
474 writea are sent to the master only
478 sub set_balanced_storage {
480 my $schema = $self->schema;
481 my $write_handler = $self->schema->storage->balancer;
483 $schema->storage->read_handler($write_handler);
486 =head2 around: txn_do ($coderef)
488 Overload to the txn_do method, which is delegated to whatever the
489 L<write_handler> is set to. We overload this in order to wrap in inside a
490 L</execute_reliably> method.
494 around 'txn_do' => sub {
495 my($txn_do, $self, $coderef, @args) = @_;
496 $self->execute_reliably(sub {$self->$txn_do($coderef, @args)});
501 Check that the master and at least one of the replicants is connected.
508 $self->master->connected &&
509 $self->pool->connected_replicants;
512 =head2 ensure_connected
514 Make sure all the storages are connected.
518 sub ensure_connected {
520 foreach my $source ($self->all_storages) {
521 $source->ensure_connected(@_);
527 Set the limit_dialect for all existing storages
533 foreach my $source ($self->all_storages) {
534 $source->limit_dialect(@_);
540 Set the quote_char for all existing storages
546 foreach my $source ($self->all_storages) {
547 $source->quote_char(@_);
553 Set the name_sep for all existing storages
559 foreach my $source ($self->all_storages) {
560 $source->name_sep(@_);
566 Set the schema object for all existing storages
572 foreach my $source ($self->all_storages) {
573 $source->set_schema(@_);
579 set a debug flag across all storages
585 foreach my $source ($self->all_storages) {
592 set a debug object across all storages
598 foreach my $source ($self->all_storages) {
599 $source->debugobj(@_);
605 set a debugfh object across all storages
611 foreach my $source ($self->all_storages) {
612 $source->debugfh(@_);
618 set a debug callback across all storages
624 foreach my $source ($self->all_storages) {
625 $source->debugcb(@_);
631 disconnect everything
637 foreach my $source ($self->all_storages) {
638 $source->disconnect(@_);
644 Due to the fact that replicants can lag behind a master, you must take care to
645 make sure you use one of the methods to force read queries to a master should
646 you need realtime data integrity. For example, if you insert a row, and then
647 immediately re-read it from the database (say, by doing $row->discard_changes)
648 or you insert a row and then immediately build a query that expects that row
649 to be an item, you should force the master to handle reads. Otherwise, due to
650 the lag, there is no certainty your data will be in the expected state.
652 For data integrity, all transactions automatically use the master storage for
653 all read and write queries. Using a transaction is the preferred and recommended
654 method to force the master to handle all read queries.
656 Otherwise, you can force a single query to use the master with the 'force_pool'
659 my $row = $resultset->search(undef, {force_pool=>'master'})->find($pk);
661 This attribute will safely be ignore by non replicated storages, so you can use
662 the same code for both types of systems.
664 Lastly, you can use the L</execute_reliably> method, which works very much like
667 For debugging, you can turn replication on/off with the methods L</set_reliable_storage>
668 and L</set_balanced_storage>, however this operates at a global level and is not
669 suitable if you have a shared Schema object being used by multiple processes,
670 such as on a web application server. You can get around this limitation by
671 using the Schema clone method.
673 my $new_schema = $schema->clone;
674 $new_schema->set_reliable_storage;
676 ## $new_schema will use only the Master storage for all reads/writes while
677 ## the $schema object will use replicated storage.
681 John Napiorkowski <john.napiorkowski@takkle.com>
683 Based on code originated by:
685 Norbert Csongrádi <bert@cpan.org>
686 Peter Siklósi <einon@einon.hu>
690 You may distribute this code under the same terms as Perl itself.