1 package DBIx::Class::Storage::DBI::Replicated;
5 use Moose::Util::TypeConstraints;
6 use DBIx::Class::Storage::DBI;
7 use DBIx::Class::Storage::DBI::Replicated::Pool;
8 use DBIx::Class::Storage::DBI::Replicated::Balancer;
12 DBIx::Class::Storage::DBI::Replicated - ALPHA Replicated database support
16 The Following example shows how to change an existing $schema to a replicated
17 storage type, add some replicated (readonly) databases, and perform reporting
20 ## Change storage_type in your schema class
21 $schema->storage_type( ['::DBI::Replicated', {balancer=>'::Random'}] );
23 ## Add some slaves. Basically this is an array of arrayrefs, where each
24 ## arrayref is database connect information
26 $schema->storage->connect_replicants(
27 [$dsn1, $user, $pass, \%opts],
28 [$dsn2, $user, $pass, \%opts],
29 [$dsn3, $user, $pass, \%opts],
32 ## Now, just use the $schema as normal
33 $schema->resultset('Source')->search({name=>'etc'});
35 ## You can force a given query to use a particular storage using the search
36 ### attribute 'force_pool'. For example:
38 my $RS = $schema->resultset('Source')->search(undef, {force_pool=>'master'});
40 ## Now $RS will force everything (both reads and writes) to use whatever was
41 ## setup as the master storage. 'master' is hardcoded to always point to the
42 ## Master, but you can also use any Replicant name. Please see:
43 ## L<DBIx::Class::Storage::Replicated::Pool> and the replicants attribute for
44 ## More. Also see transactions and L</execute_reliably> for alternative ways
45 ## to force read traffic to the master.
49 Warning: This class is marked BETA. This has been running a production
50 website using MySQL native replication as it's backend and we have some decent
51 test coverage but the code hasn't yet been stressed by a variety of databases.
52 Individual DB's may have quirks we are not aware of. Please use this in first
53 development and pass along your experiences/bug fixes.
55 This class implements replicated data store for DBI. Currently you can define
56 one master and numerous slave database connections. All write-type queries
57 (INSERT, UPDATE, DELETE and even LAST_INSERT_ID) are routed to master
58 database, all read-type queries (SELECTs) go to the slave database.
60 Basically, any method request that L<DBIx::Class::Storage::DBI> would normally
61 handle gets delegated to one of the two attributes: L</read_handler> or to
62 L</write_handler>. Additionally, some methods need to be distributed
63 to all existing storages. This way our storage class is a drop in replacement
64 for L<DBIx::Class::Storage::DBI>.
66 Read traffic is spread across the replicants (slaves) occuring to a user
67 selected algorithm. The default algorithm is random weighted.
71 The consistancy betweeen master and replicants is database specific. The Pool
72 gives you a method to validate it's replicants, removing and replacing them
73 when they fail/pass predefined criteria. Please make careful use of the ways
74 to force a query to run against Master when needed.
78 This class defines the following attributes.
82 The underlying L<DBIx::Class::Schema> object this storage is attaching
88 isa=>'DBIx::Class::Schema',
95 Contains the classname which will instantiate the L</pool> object. Defaults
96 to: L<DBIx::Class::Storage::DBI::Replicated::Pool>.
104 default=>'DBIx::Class::Storage::DBI::Replicated::Pool',
106 'create_pool' => 'new',
112 Contains a hashref of initialized information to pass to the Balancer object.
113 See L<DBIx::Class::Storage::Replicated::Pool> for available arguments.
128 The replication pool requires a balance class to provider the methods for
129 choose how to spread the query load across each replicant in the pool.
133 subtype 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
136 coerce 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
141 $type = 'DBIx::Class::Storage::DBI::Replicated::Balancer'.$type;
143 Class::MOP::load_class($type);
147 has 'balancer_type' => (
149 isa=>'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
152 default=> 'DBIx::Class::Storage::DBI::Replicated::Balancer::First',
154 'create_balancer' => 'new',
160 Contains a hashref of initialized information to pass to the Balancer object.
161 See L<DBIx::Class::Storage::Replicated::Balancer> for available arguments.
165 has 'balancer_args' => (
175 Is a <DBIx::Class::Storage::DBI::Replicated::Pool> or derived class. This is a
176 container class for one or more replicated databases.
182 isa=>'DBIx::Class::Storage::DBI::Replicated::Pool',
193 Is a <DBIx::Class::Storage::DBI::Replicated::Balancer> or derived class. This
194 is a class that takes a pool (<DBIx::Class::Storage::DBI::Replicated::Pool>)
200 isa=>'DBIx::Class::Storage::DBI::Replicated::Balancer',
202 handles=>[qw/auto_validate_every/],
207 The master defines the canonical state for a pool of connected databases. All
208 the replicants are expected to match this databases state. Thus, in a classic
209 Master / Slaves distributed system, all the slaves are expected to replicate
210 the Master's state as quick as possible. This is the only database in the
211 pool of databases that is allowed to handle write traffic.
217 isa=>'DBIx::Class::Storage::DBI',
221 =head1 ATTRIBUTES IMPLEMENTING THE DBIx::Storage::DBI INTERFACE
223 The following methods are delegated all the methods required for the
224 L<DBIx::Class::Storage::DBI> interface.
228 Defines an object that implements the read side of L<BIx::Class::Storage::DBI>.
232 has 'read_handler' => (
245 Defines an object that implements the write side of L<BIx::Class::Storage::DBI>.
249 has 'write_handler' => (
262 deployment_statements
288 This class defines the following methods.
292 L<DBIx::Class::Schema> when instantiating it's storage passed itself as the
293 first argument. So we need to massage the arguments a bit so that all the
294 bits get put into the correct places.
299 my ($class, $schema, $storage_type_args, @args) = @_;
310 Lazy builder for the L</master> attribute.
316 DBIx::Class::Storage::DBI->new($self->schema);
321 Lazy builder for the L</pool> attribute.
327 $self->create_pool(%{$self->pool_args});
330 =head2 _build_balancer
332 Lazy builder for the L</balancer> attribute. This takes a Pool object so that
333 the balancer knows which pool it's balancing.
337 sub _build_balancer {
339 $self->create_balancer(
341 master=>$self->master,
342 %{$self->balancer_args},
346 =head2 _build_write_handler
348 Lazy builder for the L</write_handler> attribute. The default is to set this to
353 sub _build_write_handler {
354 return shift->master;
357 =head2 _build_read_handler
359 Lazy builder for the L</read_handler> attribute. The default is to set this to
364 sub _build_read_handler {
365 return shift->balancer;
368 =head2 around: connect_replicants
370 All calls to connect_replicants needs to have an existing $schema tacked onto
371 top of the args, since L<DBIx::Storage::DBI> needs it.
375 around 'connect_replicants' => sub {
376 my ($method, $self, @args) = @_;
377 $self->$method($self->schema, @args);
382 Returns an array of of all the connected storage backends. The first element
383 in the returned array is the master, and the remainings are each of the
390 return grep {defined $_ && blessed $_} (
396 =head2 execute_reliably ($coderef, ?@args)
398 Given a coderef, saves the current state of the L</read_handler>, forces it to
399 use reliable storage (ie sets it to the master), executes a coderef and then
400 restores the original state.
406 $schema->resultset('User')->create({name=>$name});
407 my $user_rs = $schema->resultset('User')->find({name=>$name});
411 my $user_rs = $schema->storage->execute_reliably($reliably, 'John');
413 Use this when you must be certain of your database state, such as when you just
414 inserted something and need to get a resultset including it, etc.
418 sub execute_reliably {
419 my ($self, $coderef, @args) = @_;
421 unless( ref $coderef eq 'CODE') {
422 $self->throw_exception('Second argument must be a coderef');
425 ##Get copy of master storage
426 my $master = $self->master;
428 ##Get whatever the current read hander is
429 my $current = $self->read_handler;
431 ##Set the read handler to master
432 $self->read_handler($master);
434 ## do whatever the caller needs
436 my $want_array = wantarray;
440 @result = $coderef->(@args);
441 } elsif(defined $want_array) {
442 ($result[0]) = ($coderef->(@args));
448 ##Reset to the original state
449 $self->read_handler($current);
451 ##Exception testing has to come last, otherwise you might leave the
452 ##read_handler set to master.
455 $self->throw_exception("coderef returned an error: $@");
457 return $want_array ? @result : $result[0];
461 =head2 set_reliable_storage
463 Sets the current $schema to be 'reliable', that is all queries, both read and
464 write are sent to the master
468 sub set_reliable_storage {
470 my $schema = $self->schema;
471 my $write_handler = $self->schema->storage->write_handler;
473 $schema->storage->read_handler($write_handler);
476 =head2 set_balanced_storage
478 Sets the current $schema to be use the </balancer> for all reads, while all
479 writea are sent to the master only
483 sub set_balanced_storage {
485 my $schema = $self->schema;
486 my $write_handler = $self->schema->storage->balancer;
488 $schema->storage->read_handler($write_handler);
491 =head2 around: txn_do ($coderef)
493 Overload to the txn_do method, which is delegated to whatever the
494 L<write_handler> is set to. We overload this in order to wrap in inside a
495 L</execute_reliably> method.
499 around 'txn_do' => sub {
500 my($txn_do, $self, $coderef, @args) = @_;
501 $self->execute_reliably(sub {$self->$txn_do($coderef, @args)});
506 Check that the master and at least one of the replicants is connected.
513 $self->master->connected &&
514 $self->pool->connected_replicants;
517 =head2 ensure_connected
519 Make sure all the storages are connected.
523 sub ensure_connected {
525 foreach my $source ($self->all_storages) {
526 $source->ensure_connected(@_);
532 Set the limit_dialect for all existing storages
538 foreach my $source ($self->all_storages) {
539 $source->limit_dialect(@_);
545 Set the quote_char for all existing storages
551 foreach my $source ($self->all_storages) {
552 $source->quote_char(@_);
558 Set the name_sep for all existing storages
564 foreach my $source ($self->all_storages) {
565 $source->name_sep(@_);
571 Set the schema object for all existing storages
577 foreach my $source ($self->all_storages) {
578 $source->set_schema(@_);
584 set a debug flag across all storages
590 foreach my $source ($self->all_storages) {
597 set a debug object across all storages
603 foreach my $source ($self->all_storages) {
604 $source->debugobj(@_);
610 set a debugfh object across all storages
616 foreach my $source ($self->all_storages) {
617 $source->debugfh(@_);
623 set a debug callback across all storages
629 foreach my $source ($self->all_storages) {
630 $source->debugcb(@_);
636 disconnect everything
642 foreach my $source ($self->all_storages) {
643 $source->disconnect(@_);
649 Due to the fact that replicants can lag behind a master, you must take care to
650 make sure you use one of the methods to force read queries to a master should
651 you need realtime data integrity. For example, if you insert a row, and then
652 immediately re-read it from the database (say, by doing $row->discard_changes)
653 or you insert a row and then immediately build a query that expects that row
654 to be an item, you should force the master to handle reads. Otherwise, due to
655 the lag, there is no certainty your data will be in the expected state.
657 For data integrity, all transactions automatically use the master storage for
658 all read and write queries. Using a transaction is the preferred and recommended
659 method to force the master to handle all read queries.
661 Otherwise, you can force a single query to use the master with the 'force_pool'
664 my $row = $resultset->search(undef, {force_pool=>'master'})->find($pk);
666 This attribute will safely be ignore by non replicated storages, so you can use
667 the same code for both types of systems.
669 Lastly, you can use the L</execute_reliably> method, which works very much like
672 For debugging, you can turn replication on/off with the methods L</set_reliable_storage>
673 and L</set_balanced_storage>, however this operates at a global level and is not
674 suitable if you have a shared Schema object being used by multiple processes,
675 such as on a web application server. You can get around this limitation by
676 using the Schema clone method.
678 my $new_schema = $schema->clone;
679 $new_schema->set_reliable_storage;
681 ## $new_schema will use only the Master storage for all reads/writes while
682 ## the $schema object will use replicated storage.
686 John Napiorkowski <john.napiorkowski@takkle.com>
688 Based on code originated by:
690 Norbert Csongrádi <bert@cpan.org>
691 Peter Siklósi <einon@einon.hu>
695 You may distribute this code under the same terms as Perl itself.
699 __PACKAGE__->meta->make_immutable;