Spring boot integration Apache ftpserver detailed tutorial (recommended)

Time:2020-7-28

1、 Introduction to Apache ftpserver

Apache ftpserver is a 100% pure Java FTP server. It is designed as a complete and portable FTP server engine solution based on the currently available open protocols. Ftpserver can run independently as a Windows service or UNIX / Linux daemons, or embedded in Java applications. We also provide support for in application integration in spring and provide our distribution in the form of OSGi bundles. The default network support is based on the high-performance asynchronous IO library Apache Mina. With Mina, ftpserver can be extended to a large number of concurrent users.

2、 Features of Apache ftpserver

  • 100% pure Java, free open source, recoverable FTP server
  • Multi platform support and multi thread design.
  • User virtual directory, write permission, idle timeout and upload / download bandwidth limit support.
  • Anonymous login support.
  • Both upload and download files are recoverable.
  • Process ASCII and binary data transfer.
  • Supports IP restrictions to prohibit IP.
  • Databases and files can be used to store user data.
  • All FTP messages are customizable.
  • Implicit / explicit SSL / TLS support.
  • Mdtm support – your users can change the date and time stamp of a file.
  • Mode Z supports faster data upload / download.
  • Can easily add custom user manager, IP limiter, recorder.
  • User event notification (ftlet) can be added.

3、 Simple deployment and use of Apache ftpserver (based on windows, Linux is similar)

1. Download the corresponding version of the deployment package as required: https://mina.apache.org/ftpserver-project/downloads.html

2. Unzip the deployment package and adjust it\ users.properties And the- typical.xml configuration file


users.properties File configuration

For example, to configure a BxL user:
 #Password configuration new user
 ftpserver.user.bxl.userpassword=123456
 #Home directory, here you can customize your own home directory
 ftpserver.user.bxl.homedirectory=./res/bxl-home
 #Available to current user
 ftpserver.user.bxl.enableflag=true
 #Have the right to upload
 ftpserver.user.bxl.writepermission=true
 #The maximum number of login users is 20
 ftpserver.user.bxl.maxloginnumber=20
 #The number of users logged in with IP is 2
 ftpserver.user.bxl.maxloginperip=2
 #The idle time is 300 seconds
 ftpserver.user.bxl.idletime=300
 #The upload rate is limited to 480000 bytes per second
 ftpserver.user.bxl.uploadrate=48000000
 #The download rate is limited to 480000 bytes per second
 ftpserver.user.bxl.downloadrate=48000000

ftpd- typical.xml File configuration

<server xmlns="http://mina.apache.org/ftpserver/spring/v1"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://mina.apache.org/ftpserver/spring/v1 http://mina.apache.org/ftpserver/ftpserver-1.0.xsd">
  <listeners>
   <nio-listener name="default" port="2121">
    <ssl>
     <keystore file="./res/ftpserver.jks" password="password" />
    </ssl>
    <! -- Note: if you want to support Internet connection, you need to use passive mode, and the active mode is enabled by default -- >
    <data-connection idle-timeout="60">
     <active enabled="true" ip-check="true" />
     <!-- <passive ports="2000-2222" address="0.0.0.0" external-address="xxx.xxx.xxx.xxx" /> -->
    </data-connection>
    <! -- add IP blacklist -- >
    <blacklist>127.0.0.1</blacklist>
   </nio-listener>
  </listeners>
  
  <! -- add encrypt passwords = clear to remove password encryption -- >
  <file-user-manager file="./res/conf/users.properties" encrypt-passwords="clear" />
 </server>

3. Start and access

First start the service, open CMD and CD to bin path to execute\ ftpd.bat res/conf/ftpd- typical.xml , see the following status, indicating that the startup is successful

Test access, open the browser and input: ftp://localhost : 2121 / you will see your file directory. If you do not configure anonymous users, you will be asked to enter your user name and password user.properties Configured in

4、 Springboot integrates Apache ftpserver (emphasis)

Mode 1: deploy ftpserver service independently

This method is relatively simple, as long as the service is deployed well, and then the relevant operations are completed through ftpclien. It is the same reason that jedis accesses the redis service. There is nothing to say. Pay attention to the access mode of ftpserver. If you want to support Internet connection, you need to use passive mode.

Method 2: embed the ftpserver service into the springboot service

This method needs to be integrated with springboot, which is relatively complex, but in this way, ftpserver will be opened or destroyed with the startup or shutdown of springboot service. The specific way to use depends on your own business needs.

Briefly speaking of my implementation scheme, ftpserver supports configuration files and DB to save account information and other related configurations. If our business system needs to connect user information with FTP account information, and there are relevant business statistics, such as the time and number of files uploaded by each person in the system, the database is used to save the FTP account The number information is more convenient and flexible. I choose to use MySQL here.

Start to integrate

1. Project add dependency

//These are only dependencies related to Apache ftpserver. You can add the dependencies of springboot project itself
<dependency>
 <groupId>org.slf4j</groupId>
 <artifactId>slf4j-log4j12</artifactId>
 <version>1.7.25 </version>
</dependency>
<dependency>
 <groupId>org.apache.ftpserver</groupId>
 <artifactId>ftpserver-core</artifactId>
 <version>1.1.1</version>
</dependency>
<dependency>
 <groupId>org.apache.ftpserver</groupId>
 <artifactId>ftplet-api</artifactId>
 <version>1.1.1</version>
</dependency>

<dependency>
 <groupId>org.apache.mina</groupId>
 <artifactId>mina-core</artifactId>
 <version>2.0.16</version>
</dependency>

 

2. The database table is used to save the relevant account information (you can manually add several items for testing). Please refer to the specific field meaning users.properties File configuration (imagine that every registered user in our system can add an FTP to it_ User information, which is used to specify the user’s upload data, etc.)


CREATE TABLE FTP_USER (  
 userid VARCHAR(64) NOT NULL PRIMARY KEY,  
 userpassword VARCHAR(64),  
 homedirectory VARCHAR(128) NOT NULL,    
 enableflag BOOLEAN DEFAULT TRUE, 
 writepermission BOOLEAN DEFAULT FALSE,  
 idletime INT DEFAULT 0,    
 uploadrate INT DEFAULT 0,    
 downloadrate INT DEFAULT 0,
 maxloginnumber INT DEFAULT 0,
 maxloginperip INT DEFAULT 0
);

3. Configure ftpserver and provide init(), start(), stop() methods of ftpserver

import com.mysql.cj.jdbc.MysqlDataSource;
import com.talkingdata.tds.ftpserver.plets.MyFtpPlet;
import org.apache.commons.io.IOUtils;
import org.apache.ftpserver.DataConnectionConfigurationFactory;
import org.apache.ftpserver.FtpServer;
import org.apache.ftpserver.FtpServerFactory;
import org.apache.ftpserver.ftplet.FtpException;
import org.apache.ftpserver.ftplet.Ftplet;
import org.apache.ftpserver.listener.Listener;
import org.apache.ftpserver.listener.ListenerFactory;
import org.apache.ftpserver.ssl.SslConfigurationFactory;
import org.apache.ftpserver.usermanager.ClearTextPasswordEncryptor;
import org.apache.ftpserver.usermanager.DbUserManagerFactory;
import org.apache.ftpserver.usermanager.PropertiesUserManagerFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.stereotype.Component;
import javax.sql.DataSource;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
/**
 *Note: the class marked with @ configuration will be added to the IOC container, and all methods annotated with @ bean in the class will be dynamically proxied, so calling this method will return the same instance.
 *FTP service access address:
 *  ftp://localhost:3131/
 */
@Configuration("MyFtp")
public class MyFtpServer {
 private static final Logger logger = LoggerFactory.getLogger(MyFtpServer.class);
 //The data source can be directly injected after configuring the springboot
 @Autowired
 private DataSource dataSource;
 protected FtpServer server;
 //Here we use the spring feature of loading @ configuration to complete the initialization of FTP server
 public MyFtpServer(DataSource dataSource) {
  this.dataSource = dataSource;
  initFtp();
  logger.info("Apache ftp server is already instantiation complete!");
 }
 /**
  * ftp server init
  * @throws IOException
  */
 public void initFtp() {
  FtpServerFactory serverFactory = new FtpServerFactory();
  ListenerFactory listenerFactory = new ListenerFactory();
  //1. Set service port
  listenerFactory.setPort(3131);
  //2. Set the interface range of data upload in passive mode, and the cloud server needs to open the ports of corresponding interval to the client
  DataConnectionConfigurationFactory dataConnectionConfFactory = new DataConnectionConfigurationFactory();
  dataConnectionConfFactory.setPassivePorts("10000-10500");
  listenerFactory.setDataConnectionConfiguration(dataConnectionConfFactory.createDataConnectionConfiguration());
  //3. Add SSL security configuration
//  SslConfigurationFactory ssl = new SslConfigurationFactory();
//  ssl.setKeystoreFile(new File("src/main/resources/ftpserver.jks"));
//  ssl.setKeystorePassword("password");
  //ssl.setSslProtocol("SSL");
  // set the SSL configuration for the listener
//  listenerFactory.setSslConfiguration(ssl.createSslConfiguration());
//  listenerFactory.setImplicitSsl(true);
  //4. Replace default listener
  Listener listener = listenerFactory.createListener();
  serverFactory.addListener("default", listener);
  //5. Configure custom user events
  Map<String, Ftplet> ftpLets = new HashMap();
  ftpLets.put("ftpService", new MyFtpPlet());
  serverFactory.setFtplets(ftpLets);
  //6. Read user's configuration information
  //Note: the configuration file is located in the resources directory. If the project uses the built-in container to publish in jar package, ftpserver cannot directly read the configuration file in jar package directly.
  //Solution: copy the file to the specified directory (specified in this article to the root directory), and then ftpserver can read it.
//  PropertiesUserManagerFactory userManagerFactory = new PropertiesUserManagerFactory();
//  String tempPath = System.getProperty("java.io.tmpdir") + System.currentTimeMillis() + ".properties";
//  File tempConfig = new File(tempPath);
//  ClassPathResource resource = new ClassPathResource("users.properties");
//  IOUtils.copy(resource.getInputStream(), new FileOutputStream(tempConfig));
//  userManagerFactory.setFile(tempConfig);
//   userManagerFactory.setPasswordEncryptor (New cleartextpasswordencryptor()); // the password is in clear text
//  serverFactory.setUserManager(userManagerFactory.createUserManager());
  //6.2 store user instance based on Database
  DbUserManagerFactory dbUserManagerFactory = new DbUserManagerFactory();
  //todo....
  dbUserManagerFactory.setDataSource(dataSource);
  dbUserManagerFactory.setAdminName("admin");
  dbUserManagerFactory.setSqlUserAdmin("SELECT userid FROM FTP_USER WHERE userid='{userid}' AND userid='admin'");
  dbUserManagerFactory.setSqlUserInsert("INSERT INTO FTP_USER (userid, userpassword, homedirectory, " +
    "enableflag, writepermission, idletime, uploadrate, downloadrate) VALUES " +
    "('{userid}', '{userpassword}', '{homedirectory}', {enableflag}, " +
    "{writepermission}, {idletime}, uploadrate}, {downloadrate})");
  dbUserManagerFactory.setSqlUserDelete("DELETE FROM FTP_USER WHERE userid = '{userid}'");
  dbUserManagerFactory.setSqlUserUpdate("UPDATE FTP_USER SET userpassword='{userpassword}',homedirectory='{homedirectory}',enableflag={enableflag},writepermission={writepermission},idletime={idletime},uploadrate={uploadrate},downloadrate={downloadrate},maxloginnumber={maxloginnumber}, maxloginperip={maxloginperip} WHERE userid='{userid}'");
  dbUserManagerFactory.setSqlUserSelect("SELECT * FROM FTP_USER WHERE userid = '{userid}'");
  dbUserManagerFactory.setSqlUserSelectAll("SELECT userid FROM FTP_USER ORDER BY userid");
  dbUserManagerFactory.setSqlUserAuthenticate("SELECT userid, userpassword FROM FTP_USER WHERE userid='{userid}'");
  dbUserManagerFactory.setPasswordEncryptor(new ClearTextPasswordEncryptor());
  serverFactory.setUserManager(dbUserManagerFactory.createUserManager());
  //7. FTP server instantiation
  server = serverFactory.createServer();
 }
 /**
  * ftp server start
  */
 public void start(){
  try {
   server.start();
   logger.info("Apache Ftp server is starting!");
  }catch(FtpException e) {
   e.printStackTrace();
  }
 }
 /**
  * ftp server stop
  */
 public void stop() {
  server.stop();
  logger.info("Apache Ftp server is stoping!");
 }
}

4. Configure the listener to start ftpserver when the spring container is started, and stop ftpserver when the spring container is destroyed

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.context.WebApplicationContext;
import org.springframework.web.context.support.WebApplicationContextUtils;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;
@WebListener
public class FtpServerListener implements ServletContextListener {
 private static final Logger logger = LoggerFactory.getLogger(MyFtpServer.class);
 private static final String SERVER_NAME="FTP-SERVER";
 @Autowired
 private MyFtpServer server;
 //Call the method stop FTP server when the container is closed
 public void contextDestroyed(ServletContextEvent sce) {
//  WebApplicationContext ctx= WebApplicationContextUtils.getWebApplicationContext(sce.getServletContext());
//  MyFtpServer server=(MyFtpServer)ctx.getServletContext().getAttribute(SERVER_NAME);
  server.stop();
  sce.getServletContext().removeAttribute(SERVER_NAME);
  logger.info("Apache Ftp server is stoped!");
 }
 //Container initialization calls the method start ftpserver
 public void contextInitialized(ServletContextEvent sce) {
//  WebApplicationContext ctx= WebApplicationContextUtils.getWebApplicationContext(sce.getServletContext());
//  MyFtpServer server=(MyFtpServer) ctx.getBean("MyFtp");
  sce.getServletContext().setAttribute(SERVER_NAME,server);
  try {
   //The project was loaded when it started
   server.start();
   logger.info("Apache Ftp server is started!");
  } catch (Exception e){
   e.printStackTrace();
   throw new RuntimeException("Apache Ftp server start failed!", e);
  }
 }
}

5. Implement some custom user events by inheriting the defaultftlet abstract class (I’m just giving an example here)

import org.apache.ftpserver.ftplet.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
public class MyFtpPlet extends DefaultFtplet {
 private static final Logger logger = LoggerFactory.getLogger(MyFtpPlet.class);
 @Override
 public FtpletResult onUploadStart(FtpSession session, FtpRequest request)
   throws FtpException, IOException {
  //Get the upload path of the uploaded file
  String path = session.getUser().getHomeDirectory();
  //Get upload user
  String name = session.getUser().getName();
  //Get upload file name
  String filename = request.getArgument();
  logger.info (user: '{}', upload file to directory '{}', file name: '{}', status: start uploading ~ ", name, path, file name);
  return super.onUploadStart(session, request);
 }
 @Override
 public FtpletResult onUploadEnd(FtpSession session, FtpRequest request)
   throws FtpException, IOException {
  //Get the upload path of the uploaded file
  String path = session.getUser().getHomeDirectory();
  //Get upload user
  String name = session.getUser().getName();
  //Get upload file name
  String filename = request.getArgument();
  logger.info ("user: '{}', upload file to directory: '{}', file name: '{}, status: success! '", name, path, filename);
  return super.onUploadEnd(session, request);
 }
 @Override
 public FtpletResult onDownloadStart(FtpSession session, FtpRequest request) throws FtpException, IOException {
  //todo servies...
  return super.onDownloadStart(session, request);
 }
 @Override
 public FtpletResult onDownloadEnd(FtpSession session, FtpRequest request) throws FtpException, IOException {
  //todo servies...
  return super.onDownloadEnd(session, request);
 }
}

6. Configure the access of spring boot static resources

import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.ResourceHandlerRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
@Configuration
public class FtpConfig implements WebMvcConfigurer {
 @Override
 public void addResourceHandlers(ResourceHandlerRegistry registry) {
  //The OS can be used to determine
  String os = System.getProperty("os.name");
  //Linux settings
//  registry.addResourceHandler("/ftp/**").addResourceLocations("file:/home/pic/");
  //Windows settings
  //The first method sets the access path prefix, and the second method sets the resource path. You can specify both the project classpath path and other non project paths
  registry.addResourceHandler("/ftp/**").addResourceLocations("file:D:\apache-ftpserver-1.1.1\res\bxl-home\");
  registry.addResourceHandler("/static/**").addResourceLocations("classpath:/static/");
 }
}

7. The above six steps have completed the configuration of ftpserver. With the start of the springboot project, the ftpserver service will be opened. Here, we will paste the util for client access, and you can package it by yourself.

import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;
import org.apache.commons.net.ftp.FTPSClient;
import java.io.*;
public class FtpClientUtil {
 //FTP server IP address
 private static String FTP_ADDRESS = "localhost";
 //Port number
 private static int FTP_PORT = 3131;
 //User name
 private static String FTP_USERNAME = "bxl";
 //Password
 private static String FTP_PASSWORD = "123456";
 //Relative path
 private static String FTP_BASEPATH = "";
 public static boolean uploadFile(String remoteFileName, InputStream input) {
  boolean flag = false;
  FTPClient ftp = new FTPClient();
  ftp.setControlEncoding("UTF-8");
  try {
   int reply;
   ftp.connect (FTP_ ADDRESS, FTP_ Port); // connect to FTP server
   ftp.login (FTP_ USERNAME, FTP_ Password); // login
   reply = ftp.getReplyCode();
   System.out.println ("login FTP service return status code is" + reply) ";
   if (!FTPReply.isPositiveCompletion(reply)) {
    ftp.disconnect();
    return flag;
   }
   ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
   //Set to passive mode
   ftp.enterLocalPassiveMode();
   ftp.makeDirectory(FTP_BASEPATH);
   ftp.changeWorkingDirectory(FTP_BASEPATH);
   //Originfilepath is the file name of the uploaded file. It is recommended to use the generated unique name. It is better to transcode the Chinese name
   boolean a = ftp.storeFile(remoteFileName, input);
//   boolean a = ftp.storeFile(new String(remoteFileName.getBytes(),"iso-8859-1"),input);
   System.out.println ("the original file name to be uploaded is" + remotefilename + ", and the upload result is" + a ");
   input.close();
   ftp.logout();
   flag = true;
  } catch (IOException e) {
   e.printStackTrace();
  } finally {
   if (ftp.isConnected()) {
    try {
     ftp.disconnect();
    } catch (IOException ioe) {
    }
   }
  }
  return flag;
 }
// public static Boolean uploadFile(String remoteFileName, InputStream inputStream, String ftpAddress, int ftpPort,
//          String ftpName, String ftpPassWord, String ftpBasePath) {
//  FTP_ADDRESS = ftpAddress;
//  FTP_PORT = ftpPort;
//  FTP_USERNAME = ftpName;
//  FTP_PASSWORD = ftpPassWord;
//  FTP_BASEPATH = ftpBasePath;
//  uploadFile(remoteFileName,inputStream);
//  return true;
// }
 public static boolean deleteFile(String filename) {
  boolean flag = false;
  FTPClient ftpClient = new FTPClient();
  try {
   //Connect to FTP server
   ftpClient.connect(FTP_ADDRESS, FTP_PORT);
   //Login to FTP server
   ftpClient.login(FTP_USERNAME, FTP_PASSWORD);
   //Verify that the FTP server is logged in successfully
   int replyCode = ftpClient.getReplyCode();
   if (!FTPReply.isPositiveCompletion(replyCode)) {
    return flag;
   }
   //Switch FTP directory
   ftpClient.changeWorkingDirectory(FTP_BASEPATH);
   ftpClient.dele(filename);
   ftpClient.logout();
   flag = true;
  } catch (Exception e) {
   e.printStackTrace();
  } finally {
   if (ftpClient.isConnected()) {
    try {
     ftpClient.logout();
    } catch (IOException e) {
    }
   }
  }
  return flag;
 }
 public static boolean downloadFile(String filename, String localPath) {
  boolean flag = false;
//  FTPSClient ftpClient = new FTPSClient("TLS", true);
  FTPClient ftpClient = new FTPClient();
  try {
   //Connect to FTP server
   ftpClient.connect(FTP_ADDRESS, FTP_PORT);
   //Login to FTP server
   ftpClient.login(FTP_USERNAME, FTP_PASSWORD);
   //Verify that the FTP server is logged in successfully
   int replyCode = ftpClient.getReplyCode();
   if (!FTPReply.isPositiveCompletion(replyCode)) {
    return flag;
   }
   //Switch FTP directory
   ftpClient.changeWorkingDirectory(FTP_BASEPATH);
   //This is the demo method. Normally, you should query the file name in the database
   FTPFile[] ftpFiles = ftpClient.listFiles();
   for (FTPFile file : ftpFiles) {
    if (filename.equalsIgnoreCase(file.getName())) {
     File localFile = new File(localPath + "/" + file.getName());
     OutputStream os = new FileOutputStream(localFile);
     ftpClient.retrieveFile(file.getName(), os);
     os.close();
    }
   }
   ftpClient.logout();
   flag = true;
   System.out.println ("file download completed)!!! "";
  } catch (Exception e) {
   e.printStackTrace();
  } finally {
   if (ftpClient.isConnected()) {
    try {
     ftpClient.logout();
    } catch (IOException e) {
    }
   }
  }
  return flag;
 }
}

5、 Summary

At this point, all the configuration has been completed, and our business system also takes on a role, that is, FTP server. The whole configuration is not added with SSL / TLS security mechanism. If you are interested, you can study it yourself. In my code, I just notice that when accessing through the client, you need to use ftpscreet instead of ftpcreet. Of course, you need to configure your own ftpserver.jks File, that is, Java key store. Baidu next how to generate, very simple Oh!