FTP

Once a user has been created in the portal as described above, you may set up the logon process to use public/private key. As part of the creation of a user, an FTP start directory is created. This will be relative to the root of the FTP directory, with the account directory intervening. For example, the root directory of a “bridgegate” user will default to “/bridgegate”. If this user is assigned to the “samples” account, on the system, the user home directory will be <FTP_ROOT>/accounts/samples/bridgegate. The home directory or change be changed by updating the Start Path on the User screen in the portal.  If you update the start path to  test123 the user home directory will be <FTP_ROOT>/accounts/samples/test123

To allow this user to logon using a public/private key, the public key (OpenSSH Format), you must create a directory under this user’s root / start path FTP directory called “.ssh”. In this directory, create a file called “authorized_keys”. This file contains all the public keys authorized for this user to connect to the BridgeGate SSH (SFT) server. Each public key must be in OpenSSH format, and there must be one per line. You must press the enter key at the end of each line.

To connect to this server using a BridgeGate workflow, configure the SendData workflow item to use the FTP adapter and seleect the “SSH” option from the options button. This will enable two text boxes for entering the private key file and the password. Enter the appropriate private key information and leave the password field blank.

You can manually change the default FTP Server port by editing the /bridgegate/bin/prefs.XML  file

Search for the port tag and update the port value then restart the ftp server.   You can also use the UI.  Review the FTP Server documentation for details

<server_list>                         <port>2121</port>

Note you can get a change in IP for PASV FTP Host if your port range has been limited.

When using PASV FTP you need to provide a large enough port range to accommodate the max concurrent usage.   Some Operating systems like Windows are slow to release unused ports that will cause the range to be exhausted causing the system to change internal IP address to create a new port range.

If you use a Wild Card pattern in the File Name Mask field for File or FTP Get Data types, a transaction/workflow will be instantiated for every file meeting the wild card pattern.  For example, if you use FTM*.txt in the File Name Mask and the following files exist (FTM01.txt, FTM02.txt, FTM03.txt), three transactions/workflows will be generated.  This happens regardless of where the Get Data is located in the workflow.

FTP over SSH timeout is set using the following.

Go to the BridgeGate.xml and add the following TIMEOUT element to the existing FTP_CONFIG elements.

FTP_SSH_CONFIG TIMEOUT=”60000”
FTP_CONFIG TIMEOUT=”60000”

Getting the following exception when tried to upload a file.

 [2015-05-27 10:36:17,640] INFO  com.oidev.ftp.server.BGFileReceipt – Exception connection to BG Server Could not obtain connection to any of these urls: fe80:0:0:0:241a:9771:5f8:7c08%15:1099 and discovery failed with error: javax.naming.CommunicationException: Receive timed out [Root exception is java.net.SocketTimeoutException: Receive timed out]

[2015-05-27 10:36:17,701] INFO  com.oidev.ftp.server.BGPluginLogin – start of upload detected

[2015-05-27 10:36:17,701] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:17 EDT 2015!  [41:ftps411387:96.243.230.33] WROTE: *150 Opening BINARY mode data connection.  Ready to write file upload test – Copy (20).csv. (“/ftps411387/upload test – Copy (20).csv”) S T O R*

[2015-05-27 10:36:17,984] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:17 EDT 2015!  [41:ftps411387:96.243.230.33] WROTE: *226-Upload File Size:82940 bytes @ 80K/sec.*

[2015-05-27 10:36:17,985] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:17 EDT 2015!  [41:ftps411387:96.243.230.33] WROTE: *226 Transfer complete.  CRC32=A1A3D86E (“/ftps411387/upload test – Copy (20).csv”) STOR*

[2015-05-27 10:36:17,990] INFO  com.oidev.ftp.server.BGPluginLogin – end of upload detected

[2015-05-27 10:36:17,990] INFO  com.oidev.ftp.server.BGFileReceipt – RealTimeMonitor called for file = /ftps411387/upload test – Copy (20).csv user = ftps411387

[2015-05-27 10:36:18,000] DEBUG org.jnp.interfaces.NamingContext – Failed to connect to fe80:1099

javax.naming.CommunicationException: Failed to connect to server fe80:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server fe80:1099 [Root exception is java.net.UnknownHostException: fe80]]

        at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:269)

        at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1385)

        at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:594)

        at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:587)

        at javax.naming.InitialContext.lookup(Unknown Source)

        at com.oidev.bridgegate.SystemManager.getBridgeGateWorkFlow(SystemManager.java:196)

        at com.oidev.ftp.server.BGFileReceipt.endUpload(BGFileReceipt.java:46)

        at com.oidev.ftp.server.BGPluginLogin.endUpload(BGPluginLogin.java:200)

        at com.oidev.ftp.server.STOR_handler.run(STOR_handler.java:847)

        at java.lang.Thread.run(Unknown Source)

Caused by: javax.naming.ServiceUnavailableException: Failed to connect to server fe80:1099 [Root exception is java.net.UnknownHostException: fe80]

        at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:243)

        … 9 more

Caused by: java.net.UnknownHostException: fe80

        at java.net.InetAddress.getAllByName0(Unknown Source)

        at java.net.InetAddress.getAllByName(Unknown Source)

        at java.net.InetAddress.getAllByName(Unknown Source)

        at java.net.InetAddress.getByName(Unknown Source)

        at org.jnp.interfaces.TimedSocketFactory.createSocket(TimedSocketFactory.java:76)

        at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:239)

        … 9 more

[2015-05-27 10:36:18,112] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:18 EDT 2015!  [41:ftps411387:96.243.230.33] READ : *PASV *

[2015-05-27 10:36:18,112] INFO  com.oidev.ftp.server.BGPluginLogin – doCommand

[2015-05-27 10:36:18,112] INFO  com.oidev.ftp.server.BGPluginLogin – PASV

[2015-05-27 10:36:18,112] INFO  com.oidev.ftp.server.BGPluginLogin –

[2015-05-27 10:36:18,615] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:18 EDT 2015!  [41:ftps411387:96.243.230.33] WROTE: *227 Entering Passive Mode (10,67,59,14,12,38)*

[2015-05-27 10:36:18,719] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:18 EDT 2015!  [41:ftps411387:96.243.230.33] READ : *STOR upload test – Copy (21).csv*

[2015-05-27 10:36:18,719] INFO  com.oidev.ftp.server.BGPluginLogin – doCommand

[2015-05-27 10:36:18,719] INFO  com.oidev.ftp.server.BGPluginLogin – STOR

[2015-05-27 10:36:18,719] INFO  com.oidev.ftp.server.BGPluginLogin – upload test – Copy (21).csv

[2015-05-27 10:36:23,001] INFO  com.oidev.ftp.server.BGFileReceipt – Exception connection to BG Server Could not obtain connection to any of these urls: fe80:0:0:0:241a:9771:5f8:7c08%15:1099 and discovery failed with error: javax.naming.CommunicationException: Receive timed out [Root exception is java.net.SocketTimeoutException: Receive timed out]

[2015-05-27 10:36:23,031] INFO  com.oidev.ftp.server.BGPluginLogin – start of upload detected

[2015-05-27 10:36:23,031] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:23 EDT 2015!  [41:ftps411387:96.243.230.33] WROTE: *150 Opening BINARY mode data connection.  Ready to write file upload test – Copy (21).csv. (“/ftps411387/upload test – Copy (21).csv”) S T O R*

[2015-05-27 10:36:23,366] INFO  com.oidev.ftp.server.ServerSession – !Wed May 27 10:36:23 EDT 2015!  [41:ftps411387:96.243.230.33] WROTE: *226-Upload File Size:82940 bytes @ 80K/sec.*

— Answer –

It looks like it is be trying to use IPv6 addresses.  If you can, disable IPv6 on that system.  If you cannot, you will need to update the BridgeGate services to prefer IPv4. To do that, add the following option to the bg_*.conf files (pick an unused number where it says “XX”):

wrapper.java.additional.XX=-Djava.net.preferIPv4Stack=true

1) Create a dummy directory under the bridgegate\FTPServer directory. 

 I created mine as bridgegate\FTPServer\FauxRoot_DoNotDelete.

 2) Config bridgegate.xml to use this directory as the FTP root.

<FTP_SSH_CONFIG SERVER_PORT=”22″ SSH2_DSA_HOST_KEY=”” SSH2_DSA_HOST_KEY_PASSWORD=”” SSH2_RSA_HOST_KEY=”” SSH2_RSA_HOST_KEY_PASSWORD=”” VFS_ROOT=”C:\bridgegate\ftpserver\FauxRoot_DoNotDelete”/>

You can’t use a PPK file from BridgeGate.  The PPK is a PuTTY-(and related tools)-specific type of key. BridgeGate uses OpenSSH keys. You can export a .ppk file as an OpenSSH key from PuttyGen using the Conversions/Export OpenSSH Key menu option.

The command prompt client is probably executing different commands than what BridgeGate is executing (the low-level FTP protocol commands, like CWD, PASV, RETR, and the like).  The library also has to do additional parsing of the data received from the server.  

(Side note: You could run wireshark or some other network sniffing tool to see exactly what is getting sent and what we are sending, but it is likely that we are simply executing different commands in a different order than the Windows client.)

We use a third-party library (Apache Commons Net) to control the FTP connection.  This library creates the socket connections and directly implements the FTP protocol (RFC 959).