Filehub Install Guide

Anne Ramey

Installation

Note These installation instructions assume a Tomcat webserver. If not using a Tomcat server, use your webserver's method for deploying .war files.

  1. Ensure Tomcat is Running
  2. Copy the kinetic-filehub (.war file) to the /webapps directory
  3. Optionally configure the applications data directory to store configurations outside of the kinetic-filehub webapp (this will make future upgrades much easier).
  4. Navigate to the Bridgehub console (http://your-web-server/kinetic-filehub)
  5. The default admin username & password is admin/admin

Configuring the Console

Navigating to the Setup tab allows you to edit configuration details for the Bridgehub console.

The Configurator Username and Configurator Password allow you to modify the username/password combination that is needed to log into the Filehub Admin Console.

  • Log Level allows you to choose how verbose the logging will be.

    • Info (Production) will log only necessary information
    • Debug (Development) will log events such as incoming requests
    • Trace (Troubleshooting) is the most verbose logging available
  • Log Size (MB) is the size that the Current Log is allowed to be in MB.

Configuring a new Filestore

Filestores are instances of Filehub adapters that are configured to point to a specific datastore. Multiple filestores can be created in Kinetic Filehub to expose different datastores with different access credentials.

  1. From the Filestores tab of the admin console click the Add Filestore button.
  2. Select the desired filestore adapter. This example is for a Local filestore.
  3. Configure the common filestore properties:
  4. Name is a friendly label used to list filestores in the admin console (example: "My Filestore").
  5. Slug is a unique identifier used by integrating applications to select which filestore to connect to (example: "my-filestore").
  6. Configure the adapter specific properties (other adapters will have different properties):
  7. Directory specifies the fully qualified path to where file should be read and written to (example: "/Users/wally/filestore" or "C:\Users\wally\filestore").
  8. Click the Save button.

Configuring Filestore

Creating a Filestore Access Key

Filestore access keys are needed for integrating applications to sign requests to Kinetic Filehub filestores. These keys are generated for each filestore from the admin console.

  1. From the Filestores tab of the admin console, click the Edit link for My Filestore.
  2. Click on the Add Access Key button.
  3. Enter a short Description of the access key (example: "Kinetic Request CE" or "My Integrating Application").
  4. Copy the Key and Secret values, which will be necessary for the application to create signatures.

Browsing a Filestore

The Filehub admin console provides a browser for navigating a filestore. This browser is typically only used during integration development or for troubleshooting, and in this case can be helpful to verify that the Filestore has been set up correctly.

  1. From the Filestores tab of the admin console, click on the My Filestore name link.
  2. Use the Upload Files form to upload a test file to this directory.
  3. To open or download the file, click on the file link.
  4. To delete the, click on the Delete link.

Configuring Filehub Clustering Support with Cassandra

Configuring Filehub clustering allows multiple Filehub instances to be running with the same exact configuration. Any time an update happens in one Filehub instance, the other Filehub instances that are running in that cluster will automatically update the configuration details that changed. This cluster support makes it easier to run a load balancer among multiple Filehub instances. This clustering is accomplished by storing the configuration information in a Cassandra database and then having each Filehub instance poll for changes that happen on other instances in the cluster.

Setting up Cassandra

To setup Cassandra it is recommended to either use cqlsh (a command line program) or DevCenter (a GUI program).

Create your keyspace

There are multiple way to create a keyspace in Cassandra, so no one example will universally work for all environments. The following are a couple examples labeled with what situation they are used in.

Example Development Keyspace

CREATE KEYSPACE IF NOT EXISTS kinetic_filehub_development WITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor': 1 };

Example Production Keyspace

CREATE KEYSPACE IF NOT EXISTS kinetic_filehub_production WITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor': 3};

Use the keyspace that was created USE kinetic_filehub_development;

Create the tables that are needed for clustering and storing created Bridge configurations

CREATE TABLE IF NOT EXISTS cluster_events (
    action text,
    bucket text,
    content text,
    id timeuuid,
    source_node_key text,
    PRIMARY KEY ((bucket), id)
)
WITH CLUSTERING ORDER BY (id ASC)
 AND COMPACTION={
    'sstable_size_in_mb': '256',
    'tombstone_threshold': '0.05',
    'unchecked_tombstone_compaction': 'true',
    'tombstone_compaction_interval': '3600',
    'class': 'LeveledCompactionStrategy'
};

CREATE TABLE IF NOT EXISTS cluster_configuration_items (
    bucket text,
    content text,
    key text,
    PRIMARY KEY ((bucket), key)
)
WITH CLUSTERING ORDER BY (key ASC)
 AND COMPACTION={
    'sstable_size_in_mb': '256',
    'tombstone_threshold': '0.05',
    'unchecked_tombstone_compaction': 'true',
    'tombstone_compaction_interval': '3600',
    'class': 'LeveledCompactionStrategy'
};

Setting up Tomact

Connection Details in Tomcat

Specify the following Cassandra connection details as Java System Properties for the Tomcat instance running Kinetic Bridgehub

|Property Name (Bold are required)|Sample Value|Description| |com.kineticdata.filehub.clustering.contactpoints|127.0.0.1|Contact points are addresses of Cassandra nodes that the driver uses to discover the cluster topology (Read a more detailed explanation of contact points in the DataStax Documention).| |com.kineticdata.filehub.clustering.keyspace|kineticbridgehubdevelopment|The name of the keyspace that you previously created to store Bridgehub's data.| |com.kineticdata.filehub.clustering.username|auser|The username for the Cassandra instance.| |com.kineticdata.filehub.clustering.password|apassword|The password for the Cassandra instance.| |com.kineticdata.filehub.clustering.useSsl|false|true to use SSL, false if you are not.| |com.kineticdata.filehub.clustering.ssl.keystore|/path/to/client.keystore|Fully qualified path to the keystore file.| |com.kineticdata.filehub.clustering.ssl.keystore.password|secretpassword|Password for the keystore (default blank)| |com.kineticdata.filehub.clustering.ssl.truststore|/path/to/client.truststore|Fully qualified path to the truststore file.| |com.kineticdata.filehub.clustering.ssl.truststore.password|secretpassword|Password for the truststore (default blank)|

Connection details in clustering.properties file

Cassandra clustering connection details can also be defined in place of or alongside the Java System Property configuration values in a clustering.properties property file. This file should be placed in the Filehub data directory and the properties correspond to the Java System Properties with the com.kineticdata.filehub.clustering removed from the front of the property. Below is what a sample property file might look like:

contactpoints=127.0.0.1
keyspace=kinetic_filehub_development
username=a_user
password=a_password
useSsl=false
ssl.keystore=/path/to/client.keystore
ssl.keystore.password=secret_password

Restart Tomcat

Restart any tomcat instance that java properties were added to - the changes won't take affect until restart.

Structured Logging

Structured logging essentially is ensuring that a log file is a preset, consistent, and machine readable format. Structured logs are useful for enterprise log aggregation tools like Splunk, Graylog, or Elastic.

Filestore Adapters

Filestore Adapters are implemented as Java .jar files that can be deployed with the Kinetic Filehub web application .war file. The following adapters are included with the Kinetic Filehub web application, but additional adapters can be added to interact with other file storage solutions. Additionally, the source code for all of the adapters included with Kinetic Filehub is available from the Kinetic Community github organization: https://github.com/KineticCommunity

BMC® Remedy® Ars Adapter

The Filehub BMC® Remedy® Ars Adapter provides the functionality necessary for Kinetic Filehub to read and write files into Ars entry file fields.

Cloud Adapter

The Filehub Cloud Adapter provides the functionality necessary for Kinetic Filehub to read and write files into common cloud services such as Amazon S3, Microsoft Azure, and Openstack Swift.

Local Adapter

The Filehub Local Adapter provides the functionality necessary for Kinetic Filehub to read and write files into local directories. These directories may correspond to storage attached to the webserver or to shared storage mounted by the webserver.

Microsoft SharePoint Adapter

The Filehub Microsoft SharePoint Adapter provides the functionality necessary for Kinetic Filehub to read and write files into a Microsoft SharePoint Document folder.

Integrating a Non-Platform Tool with Filehub

In order to leverage Kinetic Filehub, an integrating application will need to generate pre-signed requests. The Java code snippit below shows an example using the Guava library (google commons) to generate the signed URL.

/**
 * The following method returns a signed URL that is valid for 5 seconds.  The results can be used 
 * to redirect a requester to filehub to download the file.  For example:
 * 
 * 1) End user makes request to: https://acme.com/integrating-application/photos/directory/file.png
 * 2) Integrating application determines end user should have access to the file
 * 3) Integrating application redirects the request to the result of the method below
 * 4) End user displays or downloads the file from: 
 *    https://acme.com/kinetic-filehub/employee-photos/directory/file.png
 *      ?expiration=1436541747284&key=2IQ1ISG&signature=SIGNATURE&filename=wally.png
 */
public String signUrl(
    String baseUrl,          // https://acme.com/kinetic-filehub
    String filestoreSlug,    // employee-photos
    String path,             // parent-directory/subdirectory/file.png
    String filenameOverride, // wally.png
    String accessKeyId,      // 2IQ1ISG
    String accessKeySecret   // BCSPDCO7LPF2INIR87QKIBS4KH
) {
    // Calculate the expiration to be 5 seconds from now
    Long expiration = System.currentTimeMillis()+(5*1000);
    // Build the signature
    String seed = accessKeySecret+" GET "+path+"?expiration="+expiration;
    if (filenameOverride != null) {
        seed += "&filename="+filenameOverride;
    }
    String signature = com.google.common.io.BaseEncoding.base64Url().encode(
        com.google.common.hash.Hashing.sha1()
            .hashString(seed, com.google.common.base.Charsets.UTF_8).asBytes());
    // Return the redirection path
    String result = baseUrl+"/filestores/"+filestoreSlug+"/"+path+
        "?expiration="+expiration+
        "&key="+accessKeyId+
        "&signature="+signature;
    if (filenameOverride != null) {
        result += "&filename="+
            com.google.common.net.UrlEscapers.urlFormParameterEscaper().escape(filenameOverride);
    }
    return result;
}