This tour is designed to give you an idea of what SPADE can do and how it can be set up to do that.
In order to execute the scenarios described in this tour you’ll need:
All of the Docker containers that make up this tour communicate with each other over a docker user-defined network. Therefore, the first step if setting up the tour is to create the appropriate network using the following command.
docker network create --driver bridge tour_network
To set up password and versions, which may vary between locations and releases of this document, the following commands set up environmental variables that will be used in this tour.
POSTGRES_PASSWORD=xxxxxxxx # the password of the 'postgres' account in the Postgres DB POSTGRES_FORMAT=10.1 SPADE_VERSION=4.1.5 # the version number of the SPADE being used in this tour SPADE_PASSWORD=yyyyyyyy SPADE_USER=tourist
This tour uses a Postgres database to manage its data (other data bases can be used but that is beyond the scope of this tour), and so you need to create a Postgres container. This container is created from the officially released image and it holds its own data. Therefore, if you rm the container you will need to re-create it from this step. (There are way to persiste data outside the container but, again, this is beyond the scope of this tour.)
The main passwords for the database (both the admin one and the SPADE one) are stored in read only files that the following commands will create.
SPADE_ZERO_SECRETS=${HOME}/spade.zero/run/secrets
mkdir -p ${HOME}/spade.zero/run/secrets
cat > ${SPADE_ZERO_SECRETS}/postgres_password << EOF
${POSTGRES_PASSWORD}
EOF
cat > ${SPADE_ZERO_SECRETS}/spade_password << EOF
${SPADE_PASSWORD}
EOF
These files can then be found by the container using environmental variables. These, along with other required environmental variables can be stored in a file using the following commands.
mkdir -p ${HOME}/docker/env/
cat > ${HOME}/docker/env/tour_postgres.env << EOF
POSTGRES_PASSWORD_FILE=/run/secrets/postgres_password
POSTGRES_FORMAT=${POSTGRES_FORMAT}
SPADE_PASSWORD_FILE=/run/secrets/spade_password
SPADE_VERSION=${SPADE_VERSION}
SPADE_USER=${SPADE_USER}
EOF
chmod 600 ${HOME}/docker/env/tour_postgres.env
Now the following commands will start the appropriate Postgres container and do some minor house keeping.
docker run \
--name tour_postgres \
--network=tour_network \
--volume ${HOME}/spade.zero/run:/run \
--env-file ${HOME}/docker/env/tour_postgres.env \
-d postgres
docker exec -i -t tour_postgres /bin/bash
apt-get -y update && apt-get -y upgrade
apt-get -y install curl wget
dpkg-reconfigure tzdata # Set this to be you local timezone
cat > ~/.pgpass << EOF
:::${SPADE_USER}:$(< ${SPADE_PASSWORD_FILE})
EOF
chmod 600 ~/.pgpass
exit
Then, to create the necessary database to be used by SPADE, you can run the following.
docker exec -i -t tour_postgres /bin/bash
wget -O spade_initialize_psql.sh \
http://nest.lbl.gov/projects/spade/resources/${SPADE_VERSION}/bash/initialize_psql.sh
. spade_initialize_psql.sh
rm spade_initialize_psql.sh
exit
Finally, the schema for the database needs to be loaded. The following commands do this. (The value of the SPADE_VERSION was set when you created the container.)
docker exec -i -t tour_postgres /bin/bash
curl -k -o ~/load.sql \
http://nest.lbl.gov/projects/spade/resources/${SPADE_VERSION}/psql/${POSTGRES_FORMAT}/load.sql
psql -U tourist -f ~/load.sql
exit
You are now ready to create the SPADE container that will be used for this tour. As with the Postgres container you will need to create a file to hold various credentials.
export HOST_UID=$(id -u ${USER})
export HOST_GID=$(id -g ${USER})
export HOST_FQDN=$(hostname -f)
cat > ${HOME}/docker/env/tour_spade.env << EOF
POSTGRES_HOST=tour_postgres
SPADE_PASSWORD_FILE=/opt/jboss/services/run/secrets/spade_password
SPADE_VERSION=${SPADE_VERSION}
SPADE_USER=${SPADE_USER}
HOST_UID=${HOST_UID}
HOST_GID=${HOST_GID}
HOST_FQDN=${HOST_FQDN}
HOST_USER=${USER}
EOF
chmod 600 ${HOME}/docker/env/tour_spade.env
You also need to map some of the container’s volume onto the local file system. The following captures both that mapping and environment file so they can be easily used when the container is launched.
SPADE_OPTIONS="-v ${HOME}/spade.zero/wildfly/extras:/opt/wildfly/workdir/extras \
-v ${HOME}/spade.zero/wildfly/standalone:/opt/wildfly/workdir/standalone \
-v ${HOME}/spade.zero/spade:/opt/jboss/services/spade \
-v ${HOME}/spade.zero/cache/spade:/opt/jboss/cache/spade \
-v ${HOME}/spade.zero/dropbox:/opt/jboss/data/spade/dropbox \
-v ${HOME}/spade.zero/warehouse:/opt/jboss/data/spade/warehouse \
-v ${HOME}/spade/shared/spade.zero/receiving:/opt/jboss/data/spade/receiving \
-v ${HOME}/spade/shared:/opt/jboss/data/external \
--env-file ${HOME}/docker/env/tour_spade.env"
Now the following command will start the SPADE container and do all of the necessary mapping between that container and the host system.
# The `cache` soft-link is only required to map SPADE's default locations
# to container's volumes. The `spade.xml` can be updated to use the
# container's volume directly.
mkdir -p ${HOME}/spade.zero
(cd ${HOME}/spade.zero ; \
mkdir -p wildfly/extras wildfly/standalone spade cache/spade dropbox warehouse ; \
ln -s /opt/jboss/cache/spade spade/cache )
mkdir -p ${HOME}/spade/shared/spade.zero/receiving
docker run \
--name tour_spade \
--network=tour_network \
--publish 8080:8080 \
--volume ${HOME}/spade.zero/run:/opt/jboss/services/run \
${SPADE_OPTIONS} \
-d lblnest/spade:${SPADE_VERSION}
To see the output of the SPADE container you can run the following command in a different terminal. This is a very useful way to make sure that other commands issued in this container have been successful,
docker logs -f tour_spade