This project has retired. For details please refer to its Attic page.
cas-filemgr – CAS File Manager User Guide
This document is in the process of being reorganized as part of our ongoing documentation effort. Please check back for updates.

Introduction

This document serves as a basic user's guide for the CAS File Manager project. The goal of the document is to allow users to check out, build, and install a base version of the CAS File Manager. Additional information can be found at the Filemgr User Guide Wiki Page. For advanced topics, such as configuring the File Manager to Scale and other tips and tricks, please see our Advanced Guide.

The remainder of this guide is separated into the following sections:

Download and Build OODT

Pre-Requisite: Maven 2.x installed on your machine and Java 1.6. If you have the correct versions of Java and Maven installed then proceed with the build process for OODT which will also build the File Manager component.

Mac OS X Users: Apple is now upgrading their OS to include Maven 3 by default, which is not compatible with Apache OODT's build process. For more information on Maven, see our Maven Guide. To determine which version of Maven is installed on your machine type the following command into a terminal.

>mvn --version
Apache Maven 2.2.1 (r801777; 2009-08-07 04:16:01+0900)
Java version: 1.6.0_26

If your system returns Apache Maven 3.X then you will need to install Maven 2.2.1 using the following steps: (NOTE: This will replace Maven 3 with Maven 2.2.1 as your systems default Maven version)

1. Download apache-maven-2.2.1-bin tar file from http://maven.apache.org

2. Install maven to /usr/share/

>sudo tar xzvf apache-maven-2.2.1-bin.tar.gz -C /usr/share

3. Now hook it in as the default maven.

>sudo rm /usr/bin/mvn
>sudo rm /usr/share/maven
>sudo ln -s /usr/share/apache-maven-2.2.1 /usr/share/maven
>sudo ln -s /usr/share/maven/bin/mvn /usr/bin/mvn

4. Double check the maven version

>mvn --version
Apache Maven 2.2.1 ...

Now you can proceed with the check out and build of Apache OODT


Create a Directory to Checkout Source Code:

>mkdir /usr/local/src
>cd /usr/local/src

Using a Tagged Release:

>svn co http://svn.apache.org/repos/asf/oodt/tags/{release_number}/ oodt

Using the Latest Development Trunk:

>svn co http://svn.apache.org/repos/asf/oodt/trunk/ oodt

Once the subversion command completes you should have a directory at /usr/local/src/oodt which will contain all of the source code for OODT. In order to build the project from source, issue the following commands:

>cd /usr/local/src/oodt
>mvn clean install

Once you get the Maven Build Successful message you can proceed with the installation of the CAS-FileManager.

>mkdir /usr/local/oodt
>cd filemgr/target
>tar xzvf cas-filemgr-{version#}-dist.tar.gz -C /usr/local/oodt
>cd /usr/local/oodt
>ln -s cas-filemgr-{version#}/ cas-filemgr

Our latest updated File Manager documentation has moved to the OODT wiki at OODT Filemgr User Guide




The resultant directory layout from the unpacked tarball is as follows:

bin/ etc/ logs/ doc/ lib/ policy/

A basic description of the files and subdirectories of the deployment is presented below:

Now you have a built File Manager at /usr/local/oodt/cas-filemgr. In the next section, we will discuss how you can configure the File Manager for basic operations.

Configuration in 2 Minutes or Less

The reason for entitling this section "in 2 Minutes or Less" is to show that in it's base deployment, with vary minimal configuration, we can have the File Manager in a usable state, capable of archiving products and capturing basic metadata. For the record, I haven't timed it, but its pretty fast...

We are going to set up the File Manager to use an XML-based Validation layer, and XML product policy, and a Lucene metadata Catalog. The first step is to edit the filemgr script in $FILEMGR_HOME/bin. Make the following changes:

  • Set the SERVER_PORT variable to the desired port on which you'd like to run the File Manager. Our default port is 9000.
  • Set the JAVA_HOME variable to point to the location of your installed JRE runtime. If you do not know where this is, type > which java and use that path.
  • Set the RUN_HOME variable to point to the location to which you'd like the File Manager PID file written. Typically this should default to /var/run, but not all system administrators allow users to write to /var/run.

The second step in configuration is to edit $FILEMGR_HOME/bin/filemgr-client script, making the following change:

  • Set the JAVA_HOME variable to point to the location of your installed JRE runtime.

In the third step of this configuration, you will set the File Manager's various extension points. For more information about the functionality of these extension points, see our Developer Guide. By default, the File Manager is built to use the XML-based repository manager and validation layer extension points, the Lucene-based catalog extension point, and the local data transfer interface.

Make the following changes to $FILEMGR_HOME/etc/filemgr.properties:

  • Specify the path to the directory where the File Manager will create the Lucene index and associated files by setting the org.apache.oodt.cas.filemgr.catalog.lucene.idxPath property to $FILEMGR_HOME/catalog. Make sure that this directory does NOT exist the first time you run the File Manager. If the File Manager does not find a directory at the specified location, it will create all of the necessary directory structure and ancillary files.
  • Specify the paths to the directories where the XML policy files are stored for the validation layer and for the repository manager. These paths are set by org.apache.oodt.cas.filemgr.repositorymgr.dirs and org.apache.oodt.cas.filemgr.validation.dirs respectively. The default location (and default policy files) are located at $FILEMGR_HOME/policy in the vanilla deployment of the File Manager. Note that these properties need to be fully specified URLs (e.g., they should start with file://).
Optionally, you can change the default logging properties for the CAS File Manager. This is done by editing $FILEMGR_HOME/etc/logging.properties. We have tried to select sensible defaults for the average user, but if you would like more or less information generated in the File Manager logs, you can edit the levels of different catagories of information in this proporties file. The following logging levels are available: INFO, WARNING, FINE, FINER, FINEST, and ALL.

With this last step, you have configured the File Manager. In order to test your configuration, cd to $FILEMGR_HOME/bin and type:

> ./filemgr start

This will startup the file manager XML-RPC server interface. Your File Manager is now ready to run! You can test out the file manager by running a simple ingest command using the filemgr-client command below. First create a simple text file called "blah.txt" and place it inside $FILEMGR_HOME/bin Then, create a blank metadata file for the product, using the schema or DTD provided in the CAS-Metadata project. An example XML file might be:

        <cas:metadata xmlns:cas="http://oodt.apache.org/1.0/cas">
        </cas:metadata>

Call this metadata file blah.txt.met, and place it also in $FILEMGR_HOME/bin. Then, run the command below, assuming that you started the File Manager on the default port of 9000:

> ./filemgr-client --url http://localhost:9000 --operation \
   --ingestProduct --productName blah.txt --productStructure Flat \
   --productTypeName GenericFile \ 
   --metadataFile file:///usr/local/oodt/cas-filemgr/bin/blah.txt.met \
   --refs file:///usr/local/oodt/cas-filemgr/bin/blah.txt

This command triggers the ingest process. You should see a response message that concludes with a line similar to:

      ingestProduct: Result: 3a812d86-148d-11db-a25a-f388f524a371

If you see something similar to this message, you have successfully configured the CAS File Manager, hopefully in 2 minutes or less (and even if it took a little more time, you have to admit it was by and large painless).

Conclusion

In this Basic User Guide, we have covered a number of topics, including File Manager installation, configuration, and a basic example of use. For more advanced topics, including the use of multiple catalogs and best practices for addressing scaling issues, see our Advanced Guide.