Providing direct memory access to SDL_Surface's pixels

In an attempt to make pixel access easier on SDL_Surface pixels. I have started work on SDLx::Surface. So far I have only start on the 32 bpp surfaces.

The general idea is to make Pointer Values (PV) of each pixel in the surface and place them into a 2D matrix. First I make pointer values like this:

SV * get_pixel32 (SDL_Surface *surface, int x, int y)
{
 //Convert the pixels to 32 bit 
 Uint32 *pixels = (Uint32 *)surface->pixels; 

 //Get the requested pixel  
 Uint32* u_ptr =  pixels + ( y * surface->w ) + x ; 

        SV* sv = newSVpv("a",1); //Make a temp SV* value on the go
        SvCUR_set(sv, sizeof(Uint32)); //Specify the new CUR length
 SvLEN_set(sv, sizeof(Uint32)); //Specify the LEN length
        SvPV_set(sv,(char*)u_ptr); // set the actual pixel's pointer as the memory space to use

 return sv; //make a modifiable reference using u_ptr's place as the memory :)

}

Next I loop through all the pixels and put them in a 2D array format, shown below:

AV * construct_p_matrix ( SDL_Surface *surface )
{
    AV * matrix = newAV();
     int i, j;
     for(  i =0 ; i < surface->w; i++)
      {
        AV * matrix_row = newAV();
        for( j =0 ; j < surface->h; j++)
        {
           av_push(matrix_row, get_pixel32(surface, i,j) );
        }
        av_push(matrix, newRV_noinc((SV*) matrix_row) );
      }

 return matrix;
}

You can see the complete here.

In Perl I can do a get access on this pixel using:

my $surf32_matrix = SDLx::Surface::pixel_array($screen_surface);
   print unpack 'b*', $surf32_matrix->[0][0]; # pixel value at x = 0 and y =0
#OUTPUT:
# 11111111000000000000000000000000

The structure of the PV is using Devel::Peek is :

print Dump $surf32_matrix->[0][0];
#OUTPUT:
#SV = PV(0xed0dbc) at 0xeb5344
#  REFCNT = 1
#  FLAGS = (POK,pPOK)
#  PV = 0x9e04ac "\0\0\377\0"
#  CUR = 4
#  LEN = 4

The problem is in setting the value of this pointer value. I have tried the following things with no success:

if ( SDL::Video::MUSTLOCK($screen_surface) ) {
    return if ( SDL::Video::lock_surface($screen_surface) < 0 ); #required for pixel operations
}

#USING pack

my $green = pack 'b*', '11111111000000000000000000000000';
substr( $surf32_matrix->[0][0], 0, 8 * 4, $green); #no change
#substr( $surf32_matrix->[0][0], 0, 8 * 4, 0xFF000000); segfault
substr( ${$surf32_matrix->[0][0]}, 0, 8 * 4, 0xFF000000); #no change
#$surf32_matrix->[0][0] = $green; SEGFAULT's cannot write to memory
${$surf32_matrix->[0][0]} = $green; #no change


SDL::Video::unlock_surface($screen_surface)
  if ( SDL::Video::MUSTLOCK($screen_surface) );

You can see an example here.

Any help will be greatly appreciated.