Configuring SPADE to connect to another SPADE deployment

This article explains how to connect two SPADE deployments so that they can transfer files between each other.

Pre-requisites

As this scenario demonstrates transfers between two SPADE deployments you will need two hosts. For simplicity the host on which you've been running the previous scenarios will be referred to as the "shipping" host that will be sending the data and the other one will be referred to as the "receiving" host that will be warehousing the data when it arrives. The model here is that you want to move data from your shipping host and store it in a remote warehouse. These are simply labels as all SPADE deployments are peers so can be interchanged to set up transfers in the opposite direction.

It is assumed that the nest-spade-war project has been install and running as outlined here on both the shipping and receiving hosts, and that the log output of both JBoss servers can be seen in appropriate terminals. It is essential that these deployments have different identities, i.e. different email addresses.

It is also helpful if you have at least read the Local Warehouse scenario in order to familiarize yourself with the concepts discussed there as they will be re-used here. Moreover, as this scenario builds upon the Emailing Confirmations scenario to provide data, you should work through that one first.

Also, as in that scenario, the following environmental variables need to be set on both host. They are show here being set to their standard values.

export JBOSS_HOME=${HOME}/server/jboss-as-7.1.1.Final
export SPADE_VERSION=2.2.0
export SPADE_WAR_HOME=${HOME}/nest-spade-war-${SPADE_VERSION}

Configuration

You now have two instances of SPADE to configure.

cat >> ~/.ssh/config << EOF

Host ${RECEIVING_HOST}
    User `whoami`
    IdentityFile ~/.ssh/`hostname`_scptransfer
EOF

scp ~/bin/spade_scp_wrapper.py ~/.ssh/`hostname`_scptransfer.pub ${RECEIVING_HOST}:.

    <outboundTransfer>
        <name>Transfer to Receiver</name>
        <neighbor>remote_user@remote_mail_host</neighbor>
        <location>receiving_host:spade/receiving/shipping_host</location>
        <shipper>receiving_host</shipper>
        <class>gov.lbl.nest.spade.services.impl.SCPTransfer</class>
    </outboundTransfer>

mkdir -p ~/spade/registrations/local
cp ${SPADE_WAR_HOME}/src/main/extras/examples/registration.3.xml ~/spade/registrations/local

mkdir -p ~/spade/receiving/${SHIPPING_HOST}
mkdir -p ~/bin
mv spade_scp_wrapper.py ~/bin
cd ~/.ssh
read PUBLIC_KEY < ~/${SHIPPING_HOST}_scptransfer.pub
echo from=\"${SHIPPING_HOST}\",command=\"$HOME/bin/spade_scp_wrapper.py -d $HOME/spade/receiving/${SHIPPING_HOST}\",\
no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding $PUBLIC_KEY >> authorized_keys
chmod 644 authorized_keys
unset PUBLIC_KEY
cd -

    <inboundTransfer>
        <name>Transfer From Shipper</name>
        <neighbor>local_user@local_mail_host</neighbor>
        <location>~/spade/receiving/shipping_host</location>
    </inboundTransfer>

cp ${SPADE_WAR_HOME}/src/main/extras/examples/mail.properties ~/spade/mail.properties
chmod 600 ~/spade/mail.properties

Execution

Execution of this scenario is similar to the scenario upon which it is built, namely the Emailing Confirmations scenario, except that the commands are now spread between the shipping and receiving hosts. To begin with the following commands should be run on the shipping host.

${JBOSS_HOME}/bin/jboss-cli.sh --connect --command="deploy \
    --name=spade.war ${SPADE_WAR_HOME}/target/spade-${SPADE_VERSION}.war"
mkdir -p ~/spade/dropbox/mock
cat > ~/spade/dropbox/mock/mock.5.data << EOF
put some junk in here
EOF
touch ~/spade/dropbox/mock/mock.5.tem
${SPADE_WAR_HOME}/src/main/bash/localScan

Inspect the shipping hosts log file and wait until you see the "finisher" complete. Then execute the following commands on the receiving host.

${JBOSS_HOME}/bin/jboss-cli.sh --connect --command="deploy \
    --name=spade.war ${SPADE_WAR_HOME}/target/spade-${SPADE_VERSION}.war"
${SPADE_WAR_HOME}/src/main/bash/inboundScan
# Wait for the "finisher" for this file to stop, e.g. 5 seconds
sleep 5
${SPADE_WAR_HOME}/src/main/bash/sendConfirmations
find ~/spade/warehouse -name "mock.5.*"

Now make sure the confirmations have been sent and return to the shipping host to execute the following command.

${SPADE_WAR_HOME}/src/main/bash/readEmail

The transfer between two instances of SPADE is now complete.

You should remember that all of the src/main/bash command you have been executing are only to force the examples thropugh the system. Left to itself all of these commands run periodically and you do not need to actively down anyython other than dorp files and their semaphores into the appropriate dropbox.

Cleanup

Having successfully completed this scenario you should now undeploy the application on both hosts using the following command.

${JBOSS_HOME}/bin/jboss-cli.sh --connect --command="undeploy spade.war"