New Module: MooseX::RemoteHelper (RFC)

Background

I have spent much of the last year writing and refining Remote Facades. At this point I've worked with SOAP, REST/JSON, and RPC url-form-encoded API's. One of the hardest parts I've found is dealing with the serialization of a Data Transfer Object with a Perl interface into whatever the remote is expecting. When I started I didn't know of these patterns, or really anything about these patterns. I highly recommend reading Patterns of Enterprise Application Architecture) if you want to know more about these patterns or things like Active Record and MVC.

problem

The problems I've encountered are many, including the fact most remotes are buggy or have a cludgy interface. Though there's nothing you can do about a remote api that you don't control, you can make your local API much cleaner and more native. Doing this however comes with a few challenges. One is that you have to map a local attribute name to a remote attribute name, because Perl uses underscores, and Java uses camel case, e.g postal_code and postalCode. The second problem is that many times the value of the attribute in its perl native form is not what the remote wants, e.g. perl boolean "1" remote "Y", or a DateTime object to W3C formatting. This second is not quite the same as mapping, because mapping is one to one, this translation could be turning an array into a comma separated string. The third problem, I didn't run into until after I "solved" the first , is how should I deal with nested complex objects (one's that can't be just converted to just a string).

My first naive remote facade was very procedural and simply assembled top to bottom, in part because it was based on SOAP::Lite, and in part because I had yet to figure out a better way. This resulted in a giant unwieldy if/then chain. Obviously my translations were just inline too.

The next thing I tried was using triggers to construct a request hash to pass to XML::Compile::SOAP. This worked better as the hash constructing code was kept right next to the attribute, so if I needed to modify the local or remote attribute, I could just go look at the attribute and the trigger tied to it.

After that I tried to use a map to translate from the native attribute name to the remote attribute name. This may have been more successful had it worked more like the Assembler in the Remote Facade. But ult imately since we were developing a rapidly changing API it seemed to bog me down, this is because I was changing the attributes on both sides of the mapping and thus the mapping at the same time (so at least 3 places). Here I was just manually dealing with the translation from a W3C DateTime format to the object I needed.

a solution

When I got assigned to yet another API and found myself doing yet another mapping and translation I decided that I needed to solve the problem. Enter the first iteration of MooseX::RemoteHelper. The first tie I used it with the form-url-encoded API so it was only needed for a single level of key, value pairs. .

Once I determined how to create MX::RemoteHelper it was simply a matter of using Class::MOP::Class API's to iterate all the attributes. The source of CompositeSerialization will give you some idea of how I did this.

Then I went back to apply this to a previous module, because the technique appears to be cleaner. Unfortunately I ran into a problem, this other API was a complex data structure, and how best to provide nested hashrefs and arrayrefs. Though I was now armed with Patterns I didn't know of one that would solve the problem. Fortunately a quick flip through the Gang of Four brought me to the Composite Pattern. I used this to write the recursive CompositeSerialization so that if I had a sufficiently complex nested structure I could simply create another object to deal with that. Here's a full example:

use 5.014;
use warnings;
use Data::Dumper;

package MessagePart {
    use Moose;
    use MooseX::RemoteHelper;
    with 'MooseX::RemoteHelper::CompositeSerialization';

    has array => (
        remote_name => 'SomeColonDelimitedArray',
        isa      => 'ArrayRef',
        is        => 'ro',
        serializer => sub {
            my ( $attr, $instance ) = @_;
            return join( ':', @{ $attr->get_value( $instance ) } );
        },
    );

    __PACKAGE__->meta->make_immutable;
}
    
package Message {
    use Moose;
    use MooseX::RemoteHelper;

    with 'MooseX::RemoteHelper::CompositeSerialization';

    has bool => (
        remote_name => 'Boolean',
        isa      => 'Bool',
        is        => 'ro',
        serializer => sub {
            my ( $attr, $instance ) = @_;
            return $attr->get_value( $instance ) ? 'Y' : 'N';
        },

    );

    has foo_bar => (
        remote_name => 'FooBar',
        isa      => 'Str',
        is        => 'ro',
    );

    has part => (
        isa      => 'MessagePart',
        remote_name => 'MyMessagePart',
        is        => 'ro',
    );

    __PACKAGE__->meta->make_immutable;
}

my $message
= Message->new({
    bool    => 0,
    foo_bar => 'Baz',
    part    => MessagePart->new({ array => [ qw( 1 2 3 4 ) ] }),
});

say Dumper $message->serialize

Which should give you this data structure:

Request For Comment

I've recently released a Trial version of MooseX::RemoteHelper to CPAN. I'm currently refactoring Business::CyberSource to use it, and it appears to be solid. What I'd like to know what people think of the module names, method names and any other comments they might have. I haven't been entirely sure that I've been naming things correctly while writing this, or that the code couldn't be better in other ways. If there's functionality you wish it had but doesn't let me know.

Adventures with SOAP using Perl: Part 2 ( SOAP::Data::Builder )

Start by reading the first 2 parts :

1. Part 0 Prelude (setup server.pl)
2. Part 1 SOAP::Lite

SOAP::Data::Builder is simply a wrapper around SOAP::Data and SOAP::Serializer for SOAP::Lite. I used it because it made my life easier building nested complicated SOAP objects. However for Part 2 I will simply be showing how to use it to do the same code as Part 1. Unfortunately since Part 1 is so Simple this actually makes SOAP::Data::Builder more complex than SOAP::Lite would be for this. In a future installment I will attempt to show more complex examples, but I will explain them less. Now let's take a look at the code.

#!/usr/bin/perl
use 5.010;
use strict;
use warnings;
use SOAP::Lite +trace => [ 'debug' ];
use SOAP::Data::Builder;

my $req = SOAP::Lite->new(
    readable   => 1,
    proxy      => 'http://localhost:8877',
    ns         => 'http://namesservice.thomas_bayer.com/',
);

my $sb = SOAP::Data::Builder->new;

$sb->add_elem(
    name  => 'name',
    value => 'Mark',
);

my $res = $req->getNameInfo( $sb->to_soap_data );

say '-' x 3;

unless ( defined $res->fault ) {
    say scalar $res->valueof('//country');
} else {
    say $res->fault->{faultstring};
    say $res->fault->{detail}{error};
}

As you can see this is much the same as the final code in Part 1, and if you run it, it does exactly the same thing. The first difference you'll notice is the self explanatory creation of the SOAP::Data::Builder object. After that comes the ->add_elem method call, which will simply create an element with a given element name and a value for that element.

Once you've finished adding elements to your $sb object, then you can call pass $sb->to_soap_data to the method that you're calling on SOAP::Lite, this generates the structure that SOAP::Lite needs to be able to make your request.

The only thing I didn't like about SOAP::Data::Builder is if you don't pass the right parameters to ->add_elem it will not croak or error in any way that will really tell you what went wrong. Simple patches to this can fix it.

Adventures with SOAP using Perl: Part 1 ( SOAP::Lite )

The most prevalent of SOAP libraries for Perl is SOAP::Lite it is the oldest and most documented. Though for all of its documentation it can be quite painful to figure out how to use it.

First make sure you've read Part 0 to set up the server. Once that's done let's look at the most simple way to interface with this server.

In our first example we need to send a request to getCountries, which is a method provided by the API. You can determine this by reading namesservice.wsdl and looking for the operations to see what's available. Essentially this means we need to send SOAP request with a Body of <getCountries />. First we need to import SOAP::Lite.
If you look at our SOAP::Lite import you'll notice that were are passing the arguments +trace => [ 'debug' ]. There are various levels and options for +trace, but this turns on full debug printing which will be sent to stderr. You don't normally want to have debug running in production code, but it will be useful to illustrate our examples and the request they send and receive.

Now let's look at creating an actual SOAP::Lite request object. The first option we pass in is readable => 1, adds whitespace to the request sent so that it's easier to read when you're looking at the debug output, you should not enable this in production, as it makes the request bigger, and I believe it is not considered correct SOAP as I've been told something about extra whitespace in SOAP being considered invalid. The second option is proxy => 'http://localhost:8877' This specifies the hostname and port that the HTTP request is sent to. ns => 'http://namesservice.thomas_bayer.com/' is the namespace, which you can find by looking for namespace in the namesservice.wsdl.

Now we need to actually create and send an actual request. For this trivial request we simply need to call the method that we need on the remote server and then return the object. You can see that SOAP::Lite is generating a namespace for your request to use with the XML <namesp1:getCountries xsi:nil="true">, which is just fine in this case.

Of course we want to do something with our response. Please note that I've modified the code to use 5.10, but if you want to use print instead of say this code will work fine on 5.6 and up. valueof, which is documented in SOAP::SOM, returns the first element in scalar context, and an array in array context. So in my code I've shown both. The syntax used in the parameters to valueof is XPath, so an even simpler way to call it in this case would be $res->valueof(//country); and it would do the same thing with this XML.

Next let's look at the getNameInfo method, it's a bit more complex so let's look at the XML in the XSD. Here's the snippet that is really important. This means that we need to send a request with a body that looks like ( note: you can look at the sample data in MyExampleData.pm for other names. ) Set let's take a stab at writing some Perl. There are some important differences to note from our previous script. You'll notice that I call ->getNameInfo() directly on the request object, instead of passing it as a parameter to ->call. This functions basically the same as call and it will end up making the first tag inside of body. We could have doen this in our first example as $req->getCountries; and that would have been it. Now that we've covered the slight differences in calls, let's go over the completely new things.

SOAP::Data objects are used to create any further data structures. Obviously the hash key of name defines the element name, and value defines what you want to go into it, here I have hardcoded "Mark".


If you run this code you'll notice that it returns a faultstring (among other fault properties) "operation getNameInfo for SOAP11 called with invalid data", and details the error as "element `c-gensym3' not processed at {http://namesservice.thomas_bayer.com/}getNameInfo". Now go back and look at the request, you'll see a c-gensym3 element, where did that come from? Well, SOAP::Lite will generate elements for anonymous elements but we can fix this.

The only difference between this and the previous code is that we aren't putting a \ in front of SOAP::Data. I wrote it the first way because I had seen examples of that all over the place, and could not find a solution to getting rid of the gensyms until I asked this question on stackoverflow.

 Unfortunately this is the most complex example that our server API has implemented. As an exercise to the reader I suggest Implementing a request for the method getNamesInCountry, which is no more complex but available.