Exporting to and Importing from files

A vcartridge can now be exported to a file and can be imported back from the same file into a vcartridge.

/quadstorvtl/bin/impexp can be used to manually export/import the vcartridge
The usage of /quadstorvtl/bin/impexp is as follows

/quadstorvtl/bin/impexp -e <1| 0> -v <vtl name> -p <vcartridge label> -d <export directory> -f <file name> -c <1|0>

Where
-e is the operation type. A value of 1 indicates and export to file and a value of 0 indicates an import from a file
-v is the name of the VTL which contains the vcartridge
-p is the vcartridge label
-d specifies the export directory where the files reside
-f specifies the file name to use. The vcartridge label is used if this value is not specified
-c specifies whether compression is used when exporting files. A value of 0 indicates the data is stored uncompressed and a value of 1 indicates that data shall be compressed. The default value is 1
-g specifies the pool name. When importing into vcartridges and if -n value is specified and if the vcartridge does not exist, the vcartridge is first created in the specified pool.
-n When importing vcartridges, if the vcartridge does not exist, the option specifies that the vcartridge be created first. The -g
option needs to be specified with this option
-x After a successful export operation, this exported vcartridge is deleted.

Examples

/quadstorvtl/bin/impexp -e 1 -v VTL1 -p VCART1 -d /mnt/export

In the above example vcartridge 'VCART1' in vtl 'VTL1' is exported to a file under directory /mnt/export . The data is compressed when written to the file. The files created under /mnt/export are /mnt/export/VCART1.data and /mnt/export/VCART1.meta. The VCART1.data file contains the data written by the backup application and VCART1.meta contains the metadata stored by the VTL software to import the data back into the VTL. Both the VCART1.data and VCART1.meta files are required to import the data back into a VTL vcartridge

/quadstorvtl/bin/impexp -e 1 -v VTL1 -p VCART1 -d /mnt/export -f VCART1.img

The above example is a slight variation from the previous one. Since the file name is specified, the files created are /mnt/export/VCART1.img and /mnt/export/VCART1.img.meta

/quadstorvtl/bin/impexp -e 1 -v VTL1 -p VCART1 -d /mnt/export -f VCART1.img -x

The above operation is similar to the one above, but the vcartridge is deleted after a successful export operation

/quadstorvtl/bin/impexp -e 1 -v VTL1 -p VCART1 -d /mnt/export -f VCART1.img -c 0

This example is the same as the second one, with the exception that the data is stored in /mnt/export/VCART1.img in an uncompressed form. This uncompressed file is essentially the data as received from the backup application. For example in the case of a tar backup, the /mnt/export/VCART1.img will contain the tar file(s)

/quadstorvtl/bin/impexp -e 0 -v VTL1 -p VCART1 -d /mnt/export -f VCART1.img

The above example is an import operation where in data from /mnt/export/VCART1.img will be imported into vcartridge VCART1 in VTL1. The vcartridge VCART1 must exist in VTL1 and will be overwritten with the new data.

/quadstorvtl/bin/impexp -e 0 -g POOL1 -n -v VTL1 -p VCART1 -d /mnt/export -f VCART1.img

The above example is an import operation where in data from /mnt/export/VCART1.img will be imported into vcartridge VCART1 in VTL1. If VCART1 does not exist it is first created in pool POOL1.

Exporting/Importing with an intermediate cache

The export operation can write to an intermediate files in a cache directory and when an intermediate file is fully written into a helper program can copy/upload the intermediate file to a remote location. The advantage of this
1. The intermediate file can be on the local file system with a smaller capacity
2. The intermediate file can then be uploaded to a remote directory (such as on a cloud filesystem) asynchronously
impexp usage with intermediate files is as follows

/quadstorvtl/bin/impexp -e <1| 0> -v <vtl name> -p <vcartridge label> -d <export directory> -f <file name> -c <1|0> -s <chunk size> -t <target dir> -k <cache copy program>

Where
-s is the chunk size.
-t is the target directory. The target directory is prefixed to the file name and passed to the cache copy program
-k is the program to run when an intermediate file is filled up.
The intermediate files will have a sequence number as the suffix.

Consider the following example

/quadstorvtl/bin/impexp -e 1 -v VTL1 -p VCART1 -d /mnt/export -f VCART1.img -s 512M  -t /mnt/remote -k /home/quadstor/cachecopy

In the above command
-e 1 indicates export.
-d /mnt/export indicates the directory to store the cached partial files
-f VCART1.img indicates the file prefix. Files will be created will be named VCART1.img.000001, VCART1.img.000002 and the metadata files will be created as VCART1.img.meta.0000001 etc.
-p VCART1 is the vcartridge to export
-v VTL1 is the VTL which contains VCART1
-k /home/quadstor/cachecopy is the copy command to run when an intermediate file is filled up. In the case of an import operation, the copy command is run to bring in data into an intermediate file in the local file system
-s 1M indicates the size of the each chunk. M or m indicate Megabytes, G or g indicate gigabytes or the size can be specified in bytes.

Examples are
-s 1M : 1MB chunk
-s 1G : 1GB chunk
-s 1048576 : 1MB chunk
-s 256m : 256MB chunk

-t The target directory. The parameters passed to the cache copy program are
-l <local file>
-r <remote file> -r is omitted if -t is not passed to impexp. If -t is passed to impexp then -r will be <target directory>/<file name> For example /mnt/cloud/CC.img.000001
-e 0 for import
-e 1 for export

For example
/home/quadstor/cachecopy -l /mnt/export/VCART1.img.000001 -r /mnt/remote/VCART1.img.000001 -e 1

Importing from files

In the "Jobs Management" page a new section "File export jobs" is available. Information regarding a successful export to file jobs is maintained in a database, and is listed in the "File Export Jobs" section as shown below
File Export Jobs
Clicking on the "Import" link for a job, imports the data from the file in the target directory into the corresponding vcartridge. The vcartridge must not exist in the VTL or should not contain data for the operation to succeed.

Clicking on the "Delete Jobs" submit button will delete the selected job information from the database.

Jobs management from command line

Listing running/completed jobs

/quadstorvtl/bin/impexp -l
Or
/quadstorvtl/bin/impexp -L (lists the number of intermediate files transferred)

Canceling/clearing a job

/quadstorvtl/bin/impexp -r -j <jobid>

Where jobid can be retrieved by impexp -l

List "File export jobs" information stored in the database

/quadstorvtl/bin/impexp -l -n

Importing a job

/quadstorvtl/bin/impexp -j <jobid> -n

Clearing an exported job from the database

/quadstorvtl/bin/impexp -r -j <jobid> -r

Post vault automation

After every vault of a vcartridge (when the vcartridge is moved into an import/export port), a post vault script will be called. This script can be be specified in /quadstorvtl/etc/quadstor.conf as
PostVaultCmd=<command to run>

For example
PostVaultCmd=/quadstorvtl/etc/postvault.sh
The quadstorvtl service needs to be restarted when the PostVaultCmd is specified or changed.

An example of the post vault command script is as below

#!/bin/sh
set -x
vtl=$1
pool=$2
label=$3

exportdir=/mnt/export/$pool
mkdir -p $exportdir

/quadstorvtl/bin/impexp -e 1 -v $vtl -p $label -d $exportdir -f "${vtl}_${pool}_${label}.img"