import org.objectweb.proactive.filetransfer.*;

static public FileVector FileTransfer.pushFile(Node n, File source, File destination);
static public FileVector FileTransfer.pushFile(Node n, File[] source, File[] destination);
static public FileVector FileTransfer.pullFile(Node n, File source, File destination);
static public FileVector FileTransfer.pullFile(Node n, File[] source, File[] destination);

These methods can be used to put and get files on a remote Node while the user's application is running. Note that these methods behave asynchronously, and in the case of the pullFile method, the returned File is a future. For further information on asynchronism and futures, please refer to the Asynchronous calls and futures section of this manual.

In the following example, a Node is deployed using a descriptor file. A file is then pushed from localhost@localSource to nodehost@remoteDest, using the paths specified in a java.io.File type object. Afterwards, a file is pulled from nodehost@remoteSource and saved at localhost@localDest, in the same fashion.

import org.objectweb.proactive.filetransfer.*;

pad = ProActive.getProactiveDescriptor(XML_LOCATION);

VirtualNode testVNode = pad.getVirtualNode("example");
testVNode.activate();
Node[] examplenode = testVNode.getNodes();

File localSource = new File("/local/source/path/file");
File remoteDest = new File("/remote/destination/path/file");
FileVector filePushed =FileTransfer.pushFile(examplenode[0],localSource, remoteDest);
filePushed.waitForAll();  //wait for push to finish

File remoteSource = new File("/remote/source/path/file");
File localDest = new File("/local/destination/path/file");
FileVector filePulled = FileTransfer.pullFile(examplenode[0], remoteSource, localDest);
File  file = filePulled.getFile(0); //wait for pull to finish

The File Transfer related tags, are placed inside the descriptor at three different parts (or levels).

The first one corresponds to the fileTransferDefinitions tag, which contains a list of FileTransfer definitions. A FileTransfer definition is a high level representation of the File Transfer, containing mainly the file names. It is created in such a way, that no low level information such as: hosts, protocols, prefix is present (this is the role of the low level representation). The following example shows a FileTranfer definition named example:

....
</deployment>
<fileTransferDefinitions>
   <fileTransfer id="example">
      <file src="hello.dat" dest="world.dat"/>
      <file src="hello.jar" dest="world.jar"/>
      <file src="hello.class" dest="world.class"/>
      <dir src="exampledir" dest="exampledir"/>
  </fileTransfer>
  <fileTransfer id="anotherExample">
      ...
  </fileTransfer>
  ...
</fileTransferDefinitions>
<infrastructure>
....         

The FileTransfer definitions can be referenced through their names, from the VirtualNode tags using two attributes:fileTransferDeploy and fileTransferRetrieve. The first one, corresponds to the file transfer that will take place at deployment time, and the second one corresponds to the file transfer that the user will trigger once the user application is done.

<virtualNode name="exampleVNode" fileTransferDeploy="example" fileTransferRetrieve="example"/>

All the low level information such as: hosts, username, protocols, prefix, etc... is declared inside each process. Both fileTransferDeploy and fileTransferRetrieve are specified separetly using a refid attribute. The refid can be a direct reference to a FileTransfer definition, or set using the keyword implicit. If implicit is used, then the reference will be inherited from the corresponding VirtualNode. In the following example both mechanisms (Deploy and Retrieve) reference indirectly and directly the example definition:

<processDefinition id="xyz">
  <sshProcess>
  ...  
<!-- Inside the process, the FileTransfer tag becomes an element instead of
an attribute.  This happens because FileTransfer information is process specific.
Note that the destination hostname and username can be omitted,
and implicitly inferred from the process information. -->

    <fileTransferDeploy refid="implicit"> <!-- referenceID or keyword "implicit" (inherit)-->
      <copyProtocol>processDefault, rcp, scp, pft</copyProtocol>
      <sourceInfo prefix="/home/user"/>
      <destinationInfo prefix="/tmp" hostname="foo.org" username="smith" />
    </fileTransferDeploy>

    <fileTransferRetrieve refid="example">
      <sourceInfo prefix="/tmp"/>
      <destinationInfo prefix="/home/user"/>
    </fileTransferRetrieve>
  </sshProcess>
</processDefinition>

In the example above, fileTransferDeploy has an implicit refid. This means that the File Transfer definitions used will be inherited from the VirtualNode. The first element shown inside this tag corresponds to copyProtocol. The copyProtocol tag specified the sequence of protocols that will be executed to achieve the FileTransfer at deployment time. Notice the processDefault keyword, which specifies the usage of the default copy protocol associated with this process. In the case of the example, this corresponds to an sshProcess and therefore the Secure Copy Protocol (scp) will be tried first. To complement the higher level File Transfer definition, other information can be specified as attributes in the sourceInfo and destinationInfo elements. For the case of FileTransferDeploy, these tags currently correspond to: prefix, hostname and username.

For fileTransferRetrieve, no copyProtocol needs to be specified. ProActive will use it's internal mechanism to transfer the files. This implies that no hostname or username are required.

import org.objectweb.proactive.core.descriptor.data;

public FileWrapper VirtualNode.fileTransferRetrieve();

import org.objectweb.proactive.core.descriptor.data;

pad = ProActive.getProactiveDescriptor(XML_LOCATION);

VirtualNode testVNode = pad.getVirtualNode("example");
testVNode.activate();
Node[] examplenode = testVNode.getNodes();

...

FileWrapper fw = testVNode.fileTransferRetrieve();
...
File f[]=fw.getFiles() //wait-for-files to arrive

This definitions can be referenced from a VirtualNode. They contain the most basic information of a FileTransfer:

References from the VirtualNode are made using the unique definition name.

These definitions contain more architecture specific information, and are therefore contained within the Process:

If some of this information (like username or hostname) can be inferred from the process, it is not necessary to declare it in the definition. Optionally, the information contained in the protocol can be overridden if specified.

Figure 23.1. File Transfer Design

When a FileTransfer starts, both abstract and concrete information are merged using the FileTransfer Workshop. The result of this process correspons to a sequence of CopyProtocols, as specified in the Concrete Definition.

Each CopyProtocol will be tried before the deployment takes place, until one succeeds. After one succeed are all fail, the process deployment will take place.

The File Transfer API is built on top of ProActive's active object and future file asynchronism model. When pulling or pushing a file from a Node, two service Active Objects (AO) are created. One is placed on the local machine and the otherone on the remote site. The file is then split into blocks, and transfered over the network using remote invocations between these two AO.

For a given virtualnode, a File Transfer pull will take place with all the nodes deployed from this virtualnode. The detailes of the specified file transfer will correspond to the ones present in the descriptor file.