ftp and sneakernet Are Not Your Only Options

Transferring files from one system to another is a routine operation. In some cases, the transfer is a one-shot, a single one-time operation, for example the migration to a new system. In other cases, the transfer is part of an ongoing process, and will be done repeatedly. I have had clients where file transfers were part of daily or hourly operational procedures.

Whether the file transfer requirement is a one-time operation, or part of an ongoing production commitment, there are a number of technological alternatives to accomplish the transfer.

In the case of migrations, OpenVMS system managers have access to the unique abilities of OpenVMS clusters to facilitate data migration from old systems and storage devices to the new. Attaching the new systems and/or storage to an existing OpenVMS cluster allows data to be copied to the new storage devices. The older hardware can then be disconnected at leisure, with no disruption to ongoing production operations. Disk migration will be the topic of a future column of the OpenVMS Consultant.

Using physical media, such as DLT (Digital Linear Tape), CD-ROM, or DVD has the potential for staggering bandwidth. As one of my professors in graduate school used to say, Never underestimate the bandwidth of a St. Bernard with a cask of CD-ROMs. The potential bandwidth of physical media has only increased with increased recording and packaging densities.

The convenience of data transfer over high performance networks is often an attractive option. The Internet Protocol suite's file transfer protocol, FTP, is often the first thought. However, FTP is certainly not the sole, and is often not the best alternative.

There are many reasons for restricting FTP access to systems, with security often one of the concerns. FTP requires system and network level configuration changes to permit its use. The use of FTP also may impose requirements for staging space on the sending system and receiving systems. Moreover, FTP is not the inevitable choice. This article will present two options that are often ignored when transferring data between two OpenVMS systems: C-Kermit and DECnet.

C-Kermit, written and maintained by the Kermit Project (http://www.columbia.edu/kermit) led by Frank da Cruz at Columbia University is an invaluable tool in transferring data between systems. C-Kermit can use any communications connection that resembles a bi-directional channel, be it a direct connection, a modem, or a network link. In the case of OpenVMS, the network connection may be a network connection limited to telnet traffic, a DECnet Remote Terminal connection, or, as this author has experimented, two OpenVMS systems which are only connectable via LAT over an otherwise unused Ethernet. C-Kermit can also be used when files are being transferred between an OpenVMS system and a completely different operating system. Fully scriptable, C-Kermit works with high efficiency over an extremely wide range of connection types and speeds, from 110 bps asynchronous dial-up connections to broadband-class LANs at speeds in excess of 100Mbps.

Most importantly from a network security perspective, C-Kermit does not require connectivity beyond a simple terminal connection. There is no need for complicated firewall rules. A simple gateway allowing TELNET or SSH connections will suffice. C-Kermit may be used over an SSH connection into an OpenVMS system (presently, there is no support for C-Kermit initiating an outbound SSH connection).

C-Kermit also does not require any privileges other than the ability to logon to the other computer and run a simple, non-privileged user program. In short, C-Kermit is suitable for situations where other network connections (e.g., FTP) are simply infeasible or inappropriate. It is straightforward to setup accounts which use C-Kermit to initiate or receive both inbound and outbound file transfer requests.

As I mentioned earlier, OpenVMS clusters provide the ultimate in convenience for file transfer. However, sometimes it is just not feasible to make the new system part of the existing OpenVMS cluster.

Another often overlooked facility are the file transfer capabilities provided by DECnet. Often, DECnet DAP (Data Access Protocol) file transfers are considered merely an equivalent function to the TCP/IP protocol stack's FTP. Nothing could be further from the truth. DECnet's DAP has many features not found in FTP, particularly its Remote File Access facilities. The DECnet DAP client integral to OpenVMS Record Management Services permits utilities and user written programs direct access to files that reside on a different OpenVMS system accessible via DECnet. It does not matter if the remote file is a simple, sequential BACKUP Save Set, or an indexed file accessed on a record-by-record basis.

In one client situation, I needed to migrate data from an existing OpenVMS cluster with newer, more powerful hardware and, more importantly, a completely restructured security scheme. Connecting the new OpenVMS cluster into the old OpenVMS cluster was infeasible for a number of reasons, both technical and political. However, by using DECnet Remote File Access, I was able to write a BACKUP Save Set of the old OpenVMS cluster's data disks directly to a temporary volume on the new OpenVMS cluster, and then restore the files onto the new SAN-based disk volumes in about one hour. Running on a dedicated 100Mbps LAN, DECnet can theoretically transfer nearly 45 Gbytes/hour.

In summary, there are more options for transferring data between systems than simply FTP or sneakernet. Each technical approach has its strong points. Savvy IT professionals consider the strengths and weaknesses of each technology before selecting the technology which provides the cleanest, most efficient solution.

URLs for referencing this entry

Long:	http://www.rlgsc.com/blog/openvms-consultant/ftp-and-sneakernet.html
Short:	http://rlgsc.com/r/20050709.html