1 package DBIx::Class::Storage::DBI::Replicated;
5 use Moose::Util::TypeConstraints;
6 use DBIx::Class::Storage::DBI;
7 use DBIx::Class::Storage::DBI::Replicated::Pool;
8 use DBIx::Class::Storage::DBI::Replicated::Balancer;
12 DBIx::Class::Storage::DBI::Replicated - ALPHA Replicated database support
16 The Following example shows how to change an existing $schema to a replicated
17 storage type, add some replicated (readonly) databases, and perform reporting
20 ## Change storage_type in your schema class
21 $schema->storage_type( ['::DBI::Replicated', {balancer=>'::Random'}] );
23 ## Add some slaves. Basically this is an array of arrayrefs, where each
24 ## arrayref is database connect information
26 $schema->storage->connect_replicants(
27 [$dsn1, $user, $pass, \%opts],
28 [$dsn2, $user, $pass, \%opts],
29 [$dsn3, $user, $pass, \%opts],
32 ## Now, just use the $schema as normal
33 $schema->resultset('Source')->search({name=>'etc'});
35 ## You can force a given query to use a particular storage using the search
36 ### attribute 'force_pool'. For example:
38 my $RS = $schema->resultset('Source')->search(undef, {force_pool=>'master'});
40 ## Now $RS will force everything (both reads and writes) to use whatever was
41 ## setup as the master storage. 'master' is hardcoded to always point to the
42 ## Master, but you can also use any Replicant name. Please see:
43 ## L<DBIx::Class::Storage::Replicated::Pool> and the replicants attribute for
44 ## More. Also see transactions and L</execute_reliably> for alternative ways
45 ## to force read traffic to the master.
49 Warning: This class is marked BETA. This has been running a production
50 website using MySQL native replication as it's backend and we have some decent
51 test coverage but the code hasn't yet been stressed by a variety of databases.
52 Individual DB's may have quirks we are not aware of. Please use this in first
53 development and pass along your experiences/bug fixes.
55 This class implements replicated data store for DBI. Currently you can define
56 one master and numerous slave database connections. All write-type queries
57 (INSERT, UPDATE, DELETE and even LAST_INSERT_ID) are routed to master
58 database, all read-type queries (SELECTs) go to the slave database.
60 Basically, any method request that L<DBIx::Class::Storage::DBI> would normally
61 handle gets delegated to one of the two attributes: L</read_handler> or to
62 L</write_handler>. Additionally, some methods need to be distributed
63 to all existing storages. This way our storage class is a drop in replacement
64 for L<DBIx::Class::Storage::DBI>.
66 Read traffic is spread across the replicants (slaves) occuring to a user
67 selected algorithm. The default algorithm is random weighted.
71 The consistancy betweeen master and replicants is database specific. The Pool
72 gives you a method to validate it's replicants, removing and replacing them
73 when they fail/pass predefined criteria. Please make careful use of the ways
74 to force a query to run against Master when needed.
78 This class defines the following attributes.
82 The underlying L<DBIx::Class::Schema> object this storage is attaching
88 isa=>'DBIx::Class::Schema',
95 Contains the classname which will instantiate the L</pool> object. Defaults
96 to: L<DBIx::Class::Storage::DBI::Replicated::Pool>.
104 default=>'DBIx::Class::Storage::DBI::Replicated::Pool',
106 'create_pool' => 'new',
112 Contains a hashref of initialized information to pass to the Balancer object.
113 See L<DBIx::Class::Storage::Replicated::Pool> for available arguments.
128 The replication pool requires a balance class to provider the methods for
129 choose how to spread the query load across each replicant in the pool.
133 subtype 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
136 coerce 'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
141 $type = 'DBIx::Class::Storage::DBI::Replicated::Balancer'.$type;
143 Class::MOP::load_class($type);
147 has 'balancer_type' => (
149 isa=>'DBIx::Class::Storage::DBI::Replicated::BalancerClassNamePart',
152 default=> 'DBIx::Class::Storage::DBI::Replicated::Balancer::First',
154 'create_balancer' => 'new',
160 Contains a hashref of initialized information to pass to the Balancer object.
161 See L<DBIx::Class::Storage::Replicated::Balancer> for available arguments.
165 has 'balancer_args' => (
175 Is a <DBIx::Class::Storage::DBI::Replicated::Pool> or derived class. This is a
176 container class for one or more replicated databases.
182 isa=>'DBIx::Class::Storage::DBI::Replicated::Pool',
193 Is a <DBIx::Class::Storage::DBI::Replicated::Balancer> or derived class. This
194 is a class that takes a pool (<DBIx::Class::Storage::DBI::Replicated::Pool>)
200 isa=>'DBIx::Class::Storage::DBI::Replicated::Balancer',
202 handles=>[qw/auto_validate_every/],
207 The master defines the canonical state for a pool of connected databases. All
208 the replicants are expected to match this databases state. Thus, in a classic
209 Master / Slaves distributed system, all the slaves are expected to replicate
210 the Master's state as quick as possible. This is the only database in the
211 pool of databases that is allowed to handle write traffic.
217 isa=>'DBIx::Class::Storage::DBI',
221 =head1 ATTRIBUTES IMPLEMENTING THE DBIx::Storage::DBI INTERFACE
223 The following methods are delegated all the methods required for the
224 L<DBIx::Class::Storage::DBI> interface.
228 Defines an object that implements the read side of L<BIx::Class::Storage::DBI>.
232 has 'read_handler' => (
245 Defines an object that implements the write side of L<BIx::Class::Storage::DBI>.
249 has 'write_handler' => (
262 deployment_statements
288 This class defines the following methods.
292 L<DBIx::Class::Schema> when instantiating it's storage passed itself as the
293 first argument. So we need to massage the arguments a bit so that all the
294 bits get put into the correct places.
299 my ($class, $schema, $storage_type_args, @args) = @_;
310 Lazy builder for the L</master> attribute.
316 DBIx::Class::Storage::DBI->new($self->schema);
321 Lazy builder for the L</pool> attribute.
327 $self->create_pool(%{$self->pool_args});
330 =head2 _build_balancer
332 Lazy builder for the L</balancer> attribute. This takes a Pool object so that
333 the balancer knows which pool it's balancing.
337 sub _build_balancer {
339 $self->create_balancer(
341 master=>$self->master,
342 %{$self->balancer_args},
346 =head2 _build_write_handler
348 Lazy builder for the L</write_handler> attribute. The default is to set this to
353 sub _build_write_handler {
354 return shift->master;
357 =head2 _build_read_handler
359 Lazy builder for the L</read_handler> attribute. The default is to set this to
364 sub _build_read_handler {
365 return shift->balancer;
368 =head2 around: connect_replicants
370 All calls to connect_replicants needs to have an existing $schema tacked onto
371 top of the args, since L<DBIx::Storage::DBI> needs it.
375 around 'connect_replicants' => sub {
376 my ($method, $self, @args) = @_;
377 $self->$method($self->schema, @args);
382 Returns an array of of all the connected storage backends. The first element
383 in the returned array is the master, and the remainings are each of the
390 return grep {defined $_ && blessed $_} (
396 =head2 execute_reliably ($coderef, ?@args)
398 Given a coderef, saves the current state of the L</read_handler>, forces it to
399 use reliable storage (ie sets it to the master), executes a coderef and then
400 restores the original state.
406 $schema->resultset('User')->create({name=>$name});
407 my $user_rs = $schema->resultset('User')->find({name=>$name});
411 my $user_rs = $schema->storage->execute_reliably($reliably, 'John');
413 Use this when you must be certain of your database state, such as when you just
414 inserted something and need to get a resultset including it, etc.
420 sub execute_reliably {
421 my ($self, $coderef, @args) = @_;
423 unless( ref $coderef eq 'CODE') {
424 $self->throw_exception('Second argument must be a coderef');
427 my $t0 = new Benchmark;
428 my $clone = $self->clone;
429 my $t1 = new Benchmark;
430 my $td = timediff($t1, $t0);
431 warn "----------------- the code took:",timestr($td),"\n";
435 ##Get copy of master storage
436 my $master = $self->master;
438 ##Get whatever the current read hander is
439 my $current = $self->read_handler;
441 ##Set the read handler to master
442 $self->read_handler($master);
444 ## do whatever the caller needs
446 my $want_array = wantarray;
450 @result = $coderef->(@args);
451 } elsif(defined $want_array) {
452 ($result[0]) = ($coderef->(@args));
458 ##Reset to the original state
459 $self->read_handler($current);
461 ##Exception testing has to come last, otherwise you might leave the
462 ##read_handler set to master.
465 $self->throw_exception("coderef returned an error: $@");
467 return $want_array ? @result : $result[0];
471 =head2 set_reliable_storage
473 Sets the current $schema to be 'reliable', that is all queries, both read and
474 write are sent to the master
478 sub set_reliable_storage {
480 my $schema = $self->schema;
481 my $write_handler = $self->schema->storage->write_handler;
483 $schema->storage->read_handler($write_handler);
486 =head2 set_balanced_storage
488 Sets the current $schema to be use the </balancer> for all reads, while all
489 writea are sent to the master only
493 sub set_balanced_storage {
495 my $schema = $self->schema;
496 my $write_handler = $self->schema->storage->balancer;
498 $schema->storage->read_handler($write_handler);
501 =head2 around: txn_do ($coderef)
503 Overload to the txn_do method, which is delegated to whatever the
504 L<write_handler> is set to. We overload this in order to wrap in inside a
505 L</execute_reliably> method.
509 around 'txn_do' => sub {
510 my($txn_do, $self, $coderef, @args) = @_;
511 $self->execute_reliably(sub {$self->$txn_do($coderef, @args)});
516 Check that the master and at least one of the replicants is connected.
523 $self->master->connected &&
524 $self->pool->connected_replicants;
527 =head2 ensure_connected
529 Make sure all the storages are connected.
533 sub ensure_connected {
535 foreach my $source ($self->all_storages) {
536 $source->ensure_connected(@_);
542 Set the limit_dialect for all existing storages
548 foreach my $source ($self->all_storages) {
549 $source->limit_dialect(@_);
555 Set the quote_char for all existing storages
561 foreach my $source ($self->all_storages) {
562 $source->quote_char(@_);
568 Set the name_sep for all existing storages
574 foreach my $source ($self->all_storages) {
575 $source->name_sep(@_);
581 Set the schema object for all existing storages
587 foreach my $source ($self->all_storages) {
588 $source->set_schema(@_);
594 set a debug flag across all storages
600 foreach my $source ($self->all_storages) {
607 set a debug object across all storages
613 foreach my $source ($self->all_storages) {
614 $source->debugobj(@_);
620 set a debugfh object across all storages
626 foreach my $source ($self->all_storages) {
627 $source->debugfh(@_);
633 set a debug callback across all storages
639 foreach my $source ($self->all_storages) {
640 $source->debugcb(@_);
646 disconnect everything
652 foreach my $source ($self->all_storages) {
653 $source->disconnect(@_);
659 Due to the fact that replicants can lag behind a master, you must take care to
660 make sure you use one of the methods to force read queries to a master should
661 you need realtime data integrity. For example, if you insert a row, and then
662 immediately re-read it from the database (say, by doing $row->discard_changes)
663 or you insert a row and then immediately build a query that expects that row
664 to be an item, you should force the master to handle reads. Otherwise, due to
665 the lag, there is no certainty your data will be in the expected state.
667 For data integrity, all transactions automatically use the master storage for
668 all read and write queries. Using a transaction is the preferred and recommended
669 method to force the master to handle all read queries.
671 Otherwise, you can force a single query to use the master with the 'force_pool'
674 my $row = $resultset->search(undef, {force_pool=>'master'})->find($pk);
676 This attribute will safely be ignore by non replicated storages, so you can use
677 the same code for both types of systems.
679 Lastly, you can use the L</execute_reliably> method, which works very much like
682 For debugging, you can turn replication on/off with the methods L</set_reliable_storage>
683 and L</set_balanced_storage>, however this operates at a global level and is not
684 suitable if you have a shared Schema object being used by multiple processes,
685 such as on a web application server. You can get around this limitation by
686 using the Schema clone method.
688 my $new_schema = $schema->clone;
689 $new_schema->set_reliable_storage;
691 ## $new_schema will use only the Master storage for all reads/writes while
692 ## the $schema object will use replicated storage.
696 John Napiorkowski <john.napiorkowski@takkle.com>
698 Based on code originated by:
700 Norbert Csongrádi <bert@cpan.org>
701 Peter Siklósi <einon@einon.hu>
705 You may distribute this code under the same terms as Perl itself.
709 __PACKAGE__->meta->make_immutable;