Putting a File in the Local Warehouse

Overview

This scenario show how a local file can be fed to SPADE which will then place it in the local warehouse.

Adding a Registration

For any file to be handled by SPADE it must be associated with a registration. This tells SPADE how it should handle any file that is associated with it. The following command creates a file containing the registration that will be used in this scenario.

mkdir -p ${HOME}/spade.zero/spade/registrations/local
cat > ${HOME}/spade.zero/spade/registrations/local/local_warehouse.1.xml << EOF
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<registration>
    <local_id>local.warehouse.1</local_id>
    <drop_box>
        <location>
            <directory>data/spade/dropbox</directory>
        </location>
        <pattern>.*.sem</pattern>
    </drop_box>
</registration>
EOF

In order to get SPADE to pick up this new registration it must be redeployed. This is done by the following command.

docker exec -it tour_spade \
    cp wars/spade-${SPADE_VERSION}.war \
    /opt/wildfly/standalone/deployments/spade.war

(You should see this redeploy happening in the other terminal showing the tour_spade container’s output.)

Creating data and semaphore files

Now that we have a registration loaded into SPADE, we can create a data file for it to handle. The following command does that.

cat > ${HOME}/spade.zero/dropbox/tour.1.data << EOF
This data file should be placed in the local warehouse
EOF

In order to SPADE to see this file a matching semaphore file needs to exist. This file should only ever be created once the .data has been completed and closed. The following command create the appropriate semaphore file for the above data file.

touch ${HOME}/spade.zero/dropbox/tour.1.sem

Telling SPADE to Handle a File

There are a number of way to tell SPADE about the existence of this file. For the purposes of this tour we’ll be using the spade-cli application in the SPADE container. This application should, eventually, be installed on the host system and detail of how to install spade-cli are provided elsewhere. If you have already done this then the first line of the following command can be omitted and you can directly run the spade-cli command.

docker exec -it tour_spade bash -l -c "spade-cli local_scan"

Once you have executed that command you will see SPADE processing the file in the other terminal showing the output of the SPADE container.

Once the finished task has completed you can confirm that the bundle got placed in the warehouse using the following command.

docker exec -it tour_spade bash -l -c "spade-cli placed"

Seeing the File in the Warehouse

After you see that SPADE has “finished” with the file you can see that it is in weather using the following command.

find ${HOME}/spade.zero/warehouse -name "*tour.1.*"

In the default implementation of the warehouse, files are placed in accordance with their SHA checksum. This implementation can be easily replaced to one that better suits you needs.

You will also notice that there is a .meta.xml file beside the data file. THis is the file used by SPADE to hold information about the file. You can inspect this file using the following command.

xmllint -format $(find ${HOME}/spade.zero/warehouse -name "tour.1.meta.xml")

As you can see, the default version contains no information. Again this can be easily replaced with a more useful metadata file.

NEXT STEP