Configuring SPADE to use a customized placement policy

This article explains how to configure a SPADE deployment so that data it uses a customized PlacingPolicy class rather than the default, which simply places the file in a directory based on the date contained in the files metadata.

Pre-requisites

It is assumed that the nest-spade-war project has been install and running as outlined here, and that the log output of the JBoss server can be seen in a second terminal.

It is also helpful if you have at least read the Local Warehouse scenario in order to familiarize yourself with the concepts discussed there as they will be re-used here. Moreover, as this scenario builds upon the Customized Metadata scenario to provide data, you should work through that one first.

Also, as in that scenario, the following environmental variables need to be set. They are show here being set to their standard values.

export JBOSS_HOME=${HOME}/server/jboss-as-7.1.1.Final
export SPADE_VERSION=2.2.0
export SPADE_WAR_HOME=${HOME}/nest-spade-war-${SPADE_VERSION}

Configuration

The placement of files into SPADE's warehouse is controlled by an implementation of the gov.lbl.nest.spade.policy.PlacingPolicy. To set up a customized PlacingPolicy instance, in this case the MirrorPlacing class, you will need to add the following XML to your spade.xml file as part of the assembly element, after the name element.

        <activity>
            <name>placer-inbound</name>
            <policy>
                <class>gov.lbl.nest.spade.policy.impl.MirrorPlacing</class>
               <init-param>
                   <param-name>mapping</param-name>
                   <param-value>localhost:/home/spade/spade/dropbox/mock/../../other=example</param-value>
               </init-param>
            </policy>
        </activity>

The MirrorPlacing class takes the file's original path from its metadata and uses a new path, created by using the mapping specified in the XML above, to place the file in the warehouse.

Execution

Execution of this scenario is the same as the scenario upon which it is built, namely the Customized Metadata scenario. The following commands show how to run the whole scenario.

${JBOSS_HOME}/bin/jboss-cli.sh --connect --command="deploy \
    --name=spade.war ${SPADE_WAR_HOME}/target/spade-${SPADE_VERSION}.war"
mkdir -p ~/spade/other/directory/mock
cat > ~/spade/other/directory/mock/mock.8.data << EOF
put some junk in here
EOF
echo "../../other/directory/mock/mock.8.data" > ~/spade/dropbox/mock/mock.8.nem
${SPADE_WAR_HOME}/src/main/bash/localScan
# Wait for the "finisher" for this file to stop, e.g. 5 seconds
sleep 5
${SPADE_WAR_HOME}/src/main/bash/inboundScan
# Wait for the "finisher" for this file to stop, e.g. 5 seconds
sleep 5
${SPADE_WAR_HOME}/src/main/bash/sendConfirmations
find ~/spade/warehouse -name "mock.8.*"

As you can see from the last command, the placement in the warehouse of the file is now a function of its original location on its source node.

Cleanup

Having successfully completed this scenario you should now undeploy the application using the following command.

${JBOSS_HOME}/bin/jboss-cli.sh --connect --command="undeploy spade.war"