Web Utilities

*WEB/SFTP

Interface with SFTP Server

Description

The *web/sftp utility is used to interface with a server that is running an SFTP server. It can be used to send files to, receive files from, delete files from, or get a directory listing from an SFTP server.

Note:
The *web/sftp utility uses curl - the URL data transfer command line tool. More specifically, it uses a version of curl built with SFTP support. A compatible version of the curl utility is included in the Windows installation of PxPlus. On many UNIX/Linux systems, the curl utility comes with the OS; however, it may or may not support SFTP.

On UNIX/Linux systems where you want to use *web/sftp, check that a compatible version of curl is installed and is in the path by running the curl -V command from the Command line. If the response is curl version information, this confirms that curl is installed and is in the path. If the response is an error stating that curl cannot be found, then you need to install it, or if already installed, add the location of curl to the path. Once curl is installed and is in the path, check that SFTP is listed in the supported protocols section of the output, and if listed, this confirms that *web/sftp will work. If it is not listed, you can find and install a version of curl with SFTP support by visiting https://curl.haxx.se/download.html.

(The *web/sftp utility was added in PxPlus 2017.)

Formats

1.

CALL "*web/sftp;send", server$, user$, password$, key_path$, key_passphrase$, insecure, timeout, files_to_send$, upload_dir$, case$, result$

2.

CALL "*web/sftp;receive", server$, user$, password$, key_path$, key_passphrase$, insecure, timeout, files_to_receive$, download_dir$, case$, result$

3.

CALL "*web/sftp;delete", server$, user$, password$, key_path$, key_passphrase$, insecure, timeout, files_to_delete$, case$, result$ (added in PxPlus 2018)

4.

CALL "*web/sftp;list", server$, user$, password$, key_path$, key_passphrase$, insecure, timeout, recursion, result$

Where:

server$

(Input) Contains the name of the SFTP server. It should be in the form of SERVER;SOCKET.

Example:

   sftp.myserver.com;22

If no socket number is specified, then a default SFTP port of 22 is used.

An SFTP sub-directory can be appended to the SERVER portion. See Important Note.

Example:

   sftp.myserver.com/work/sftp

Important Note:
The sub-directory path must URL encode any special URL characters (except for slashes).

Example:

"sftp.pvx.com/home/myapp/Help"+CVS("?","ASCII:URL")

Maximum length of server$ is 128 characters.

(TLS support for SFTP was added as of PxPlus 2020.)

user$

(Input) Contains the user ID. Maximum length of user$ is 64 characters.

password$

(Input) Contains the user password. If key_path$ is not null, this will be ignored. Maximum length of password$ is 64 characters. See SSH Authentication below.

key_path$

(Input) Contains the full path of the OpenSSH Private key file to be used for authentication. If present, password$ will be ignored. If null, password authentication will be used. See SSH Authentication below.

key_passphrase$

(Input) The passphrase for the OpenSSH Private key, if any. See SSH Authentication below.

insecure

(Input) If non-zero, this will avoid known_hosts checking. If zero, it will check for the server's Public key in the known_hosts file. See SSH Authentication below.

timeout

(Input) If non-zero, this parameter specifies the timeouts for communicating with an SFTP server. If zero, the timeout value is set to 60 seconds.

files_to_send$

(Input) List of the files to send to an SFTP server or a pattern matching the files to send to an SFTP server. Maximum length of files_to_send$ is 32,000 characters. See Usage Details below.

Note:
files_to_send$ cannot be null.

files_to_receive$

(Input) List of the files to receive from an SFTP server or a pattern matching the files to receive from an SFTP server. Maximum length of files_to_receive$ is 32,000 characters. See Usage Details below.

Note:
files_to_receive$ cannot be null.

files_to_delete$

(Input) List of the files to delete from an SFTP server or a pattern matching the files to delete from an SFTP server. Maximum length of files_to_delete$ is 32,000 characters. See Usage Details below.

Note:
files_to_delete$ cannot be null.

(files_to_delete$ was added in PxPlus 2018.)

upload_dir$

(Input) Contains the path and the directory name on an SFTP server to use to store files. If null, the upload directory is the root directory of the SFTP server.

download_dir$

(Input) Contains the path and the directory name on a client machine to use to store files. If null, the download directory is the working directory of the machine.

case$

(Input) Possible values are:

"U"

Converts all file and directory names to uppercase

"L"

Converts all file and directory names to lowercase

""

Does not change file and directory names

recursion

(Input) If non-zero, the directory listing returned will include the listing of all sub-directories as well. If zero, the directory listing will only list the current directory.

result$

(Output) If doing a LIST (see Format 3 above), the program loads this variable with the directory list results.

If an error is encountered while sending, receiving or listing files, then the program loads this variable with a textual description of the problem and exits with an error.

If one (or more) of the sending/receiving file names is invalid, then all remaining files that can be sent/received will be transferred, and those which cannot be transferred will be returned in the result$ field.

SSH Authentication

SFTP uses SSH for communication. SSH allows for Private/Public key authentication, as well as password authentication. Private/Public key authentication will be attempted first before password authentication.

Creating a Private/Public Key

To create a Private/Public key pair for SSH authentication, follow the steps below:

Windows

 

1.

Download and install putty (if not already installed) by visiting: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html.

 

2.

Launch puttygen.exe.

 

3.

Click the Generate button.

 

4.

Move the mouse around randomly in a blank area.

 

5.

Modify the default key comment to something like your_name@server_name.

 

6

(Optional) Specify a key passphrase for added security.

 

7.

Click the Conversions menu item and select Export OpenSSH key.

 

8.

Copy the Public key from the screen area labeled Public key for pasting into the authorized_keys file in the SSH directory. Once the key has been copied, puttygen can be closed.

UNIX/Linux

 

1.

At the Command prompt, type: ssh-keygen -t rsa -b 2048 -C "your_email@example.com".

 

2.

When prompted to "Enter a file in which to save the key," press Enter. This accepts the default file location. Alternatively, you could specify the location and filename of your key.

 

3.

(Optional) At the prompt, type a secure passphrase for added security.

 

4.

At the Command prompt, type: cat pub_key_path. Copy the output.

Adding the Public Key to the Server

Now that a Public/Private key pair has been created, add your Public key to the server's authorized keys file using the steps below, assuming you have control over the server. (If you do not have control, the server administrator will need to add your Public key.)

 

1.

Access the server.

 

2.

Navigate to the SSH directory, i.e. /root/.ssh.

 

3.

Open the authorized_keys2 file for editing.

 

4.

Append your Public key to the end of the file.

 

5.

Save the changes.

Adding the Server's Public Key to the Client

The final step is to add the server's Public key to the client machine's known_hosts file using the steps below, assuming you have control over the server. (If you do not have control over the server, obtain the server's Public key from the server administrator.)

This final step is optional. However, if you do not perform this step, then you must set the insecure option to non-zero when using *web/sftp. Keep in mind that things will not be as secure, as the client will not be verifying the identity of the server.

 

1.

Access the server.

 

2.

Navigate to the SSH configuration directory; i.e. /etc/ssh.

 

3.

At the Command prompt, type: cat pub_key_path. Copy the output. (The Public key is usually named something like ssh_host_rsa_key.pub.)

 

4.

On the client machine, open/create the known_hosts file for editing. (On UNIX/Linux, this is usually /root/.ssh/known_hosts. On Windows, this is %USERPROFILE%\AppData\Roaming\_ssh\known_hosts.)

 

5.

Navigate to the end of the file and start a new line. Then, insert the hostname or IP address, insert a space, and then append the server's Public key.

 

6.

Save the changes.

Usage Details

files_to_send$ / files_to_receive$ / files_to_delete$

Wild characters are accepted in file names (unless MSK( ) regular expression pattern matching is enabled):

 

?

Any one character

 

*

Any number of any characters

Arguments can contain more than one file name. File names should be separated using one of the following: $8A$, $0A$, $0D0A$, or ; (semi-colon).

Note:
If you have a directory or filename that includes one of the wildcard or separator characters, you cannot reference that path directly in the files list, as that character will be interpreted as a wildcard. You need to use MSK( ) mode and escape any special characters.

Example:

If the filename is "Help?", then:

     files_to_send$="Help\?|M"

A number of options may be specified after a file name. Options should be separated from the file name by a | (pipe). Options are:

Overwrite
(if the file already exists on the SFTP server)

0 = Always overwrite (default)
1 = Never overwrite

Ignored in files_to_delete$.

Recursion

R = Recursion ON 
"" = Recursion OFF (default)

In files_to_delete$, only applies to pattern matching.

If a directory is specified, it must recursively delete it regardless of this option.

Regular Expression Pattern Matching

M = Enable MSK( ) regular expression pattern matching
"" = Wildcard pattern matching (default)

 

Example:

file_name1|1;c:\tmp\*.*|0R;C:\\reports\\report[0-9]+\.(pdf|doc|docx)|M

In this example, file_name1 will not be transferred if it currently exists. The directory c:\tmp and all files with it will be transferred and will always overwrite any existing file of the same name. It will also recurse through all sub-directories of c:\tmp and transfer all files in every sub-directory. All files in c:\reports that start with the name report, end with a number and have the extension .pdf, .doc or .docx will be transferred and will always overwrite any existing file of the same name.

Note:
SFTP does not support ASCII transfer mode like FTP. All transfers are binary transfers.
SFTP does not support the Overwrite if newer option.

See Also

MSK( ) Scan String for Mask
Interface with FTP Server