IO and NiO in Java

Time:2020-8-21

At the end of the paper, we will choose NiO and RPC related materials for further study, so this paper begins to learn NiO knowledge.

 IOKnowledge review

Before learning NiO, it is necessary to review some knowledge of Io.

IOFlow in

Java program completes input and output through stream. Flow is the abstraction of production or consumption information. Flow is connected with physical devices through Java input and output. Although the physical devices connected with them are not the same, the behavior of all flows is the same. Therefore, the functions and methods of the same I / O class are applicable to all external devices. This means that an input stream can abstract multiple types of input, such as files, keyboards, or network sockets. Similarly, an output stream can be output to a console, file, or connected network.

Classification of flows

Flow can be divided into input stream and output stream. Input and output are relative to programs. Programs play two roles in using data: one is source, the other is purpose. If a program is the source of data and outputs data to the outside world, we call the data stream the output stream relative to the program. If the program is the destination of the data, we call the data stream the input stream relative to the program.

In structure, stream can be divided into byte stream and character stream. Byte stream takes byte as processing unit, character stream takes character as processing unit.

From the role, the flow can be divided into node flow and filter flow. A stream that reads and writes from a specific place is called a node stream, such as a disk or a memory area, while a filter stream takes a node stream as its input or output. A filter stream is created by using an existing input stream or output stream connection.

The input stream and output stream of byte stream are based on InputStream and OutputStream. The input and output operations of byte stream are implemented by subclasses of these two classes. Character stream is a new input and output stream based on characters after Java 1.1. The input and output of character stream is based on reader and writer.

Byte stream provides a convenient way to process the input and output of bytes, such as using byte stream to read or write binary data. Character stream provides convenience for character input and output. It adopts a unified coding standard, so it can be internationalized. It should be noted that in the underlying implementation of the computer, all input and output are in the form of bytes, and the character stream only provides specific methods for character processing.

 

Input stream

The logic of reading data is as follows:

open a stream

while more information

read information

close the stream

Ignoring exception handling, the related code implementation is roughly as follows:

InputStream input = new FileInputStream("c:\data\input-text.txt");

int data = input.read();
while(data != -1) {

    //do something with data...

    doSomethingWithData(data);

    data = input.read();

}

input.close();

 

Output stream

The logic of writing data is as follows:

open a stream

while more information

write information

close the stream

Ignoring exception handling, the related code implementation is roughly as follows:

OutputStream output = new FileOutputStream("c:\data\output-text.txt");

while(hasMoreData()) {

    int data = getMoreData();

    output.write(data);

}

output.close();

 

Class hierarchy of input stream

 

 

 Class hierarchy of output stream

 

 

 Filter flow

In the subclasses of InputStream and OutputStream, filterinputstream and filteroutputstream filter streams derive the subclasses of datainputstream and dataoutputstream.

IOLinks to streams

Input Stream Chain: write data from an external file to the program. Therefore, the first step is to construct an input stream, which is also a node stream. In order to make this stream buffered, it needs to flow from node to filter stream. Bufferedinputstream only has buffering feature, which can not meet daily needs. It also needs to have the feature of reading basic data types, which can be based on The existing filter stream is converted into another filter stream, datainputstream. At this time, it is convenient to read data from the file;

Output Stream Chain: write data to an external file. First of all, for the external file, it is fileoutputstream. In order to make this flow have the buffering feature, it needs to flow from the node to the filter stream. Bufferedoutputstream only has the buffering feature, which may not meet the daily needs. It also needs the feature of writing out the basic data type, which can be converted to other based on the existing filter stream At this time, you can easily write various data types from;

 ReaderClass hierarchy of

 

 WriterClass hierarchy of

 

So far, I’ve reviewed some of the basics of Io.

IOAnd decoration mode

Back to the IO stream link, the general code of input stream chain is as follows:

InputStream input = new DataInputStream(new BufferedInputStream(new FileInputStream("c:\data\input-text.txt")))

The general code of output stream chain is as follows:

OutputStream output = new DataOutputStream(new BufferedOutputStream(new FileOutputStream("c:\data\output-text.txt")))

In fact, the implementation mechanism of one stream connecting with another to form a stream pipeline is actually an application of decoration pattern.

 

The pattern of decoration pattern

Component: gives an abstract interface to specify the objects that are ready to receive additional responsibilities

Concrete component: defines a class that will receive additional responsibilities

Decorator: holds a component object reference and defines an interface consistent with the abstract build interface

Concrete decorator: responsible for attaching additional responsibilities to component objects

 Code implementation of decoration mode

Let’s look at the code:

public interface Component {
    void doSomething();
}
public class ConcreteComponent implements Component{
    @Override
    public void doSomething() {
        System.out.println ("function a");
    }
}
Public class decorator implements component {// key 1 defines interfaces consistent with abstract component interfaces
    Private component component; // key 2 holds the reference of component object

    public Decorator(Component component) {
        this.component = component;
    }

    @Override
    public void doSomething() {
        component.doSomething();
    }
}

The last part is the code of decoration role

public class ConcreteDecorator1 extends Decorator{
    public ConcreteDecorator1(Component component) {
        super(component);
    }

    @Override
    public void doSomething() {
        super.doSomething();
        this.doAnotherThing();
    }

    public void doAnotherThing() {
        System.out.println ("function B");
    }

}
public class ConcreteDecorator2 extends Decorator{
    public ConcreteDecorator2(Component component) {
        super(component);
    }

    @Override
    public void doSomething() {
        super.doSomething();
        this.doAnotherThing();
    }

    public void doAnotherThing() {
        System.out.println ("function C");
    }

}

For the client, only the following simple code is needed to complete the decoration of the component object concretecomponent:

Component component = new ConcreteDecorator1(new ConcreteDecorator2(new ConcreteComponent()));
component.doSomething();

IOExplanation of the corresponding decoration pattern in

The roles of datainputstream and bufferedinputstream are like concretedecorator1 and concretedecorator2 mentioned above. Filterinputstream is similar to decorator, and InputStream is a component.

In the source code of JDK:

public class FilterInputStream extends InputStream {
protected volatile InputStream in;
protected FilterInputStream(InputStream in) {
        this.in = in;}
public int read() throws IOException {
        return in.read();}

Let’s take a look at decorator

Public class decorator implements component {// key 1 defines interfaces consistent with abstract component interfaces
    Private component component; // key 2 holds the reference of component object

    public Decorator(Component component) {
        this.component = component;
    }

    @Override
    public void doSomething() {
        component.doSomething();
    }
}

At this point, we can know how IO is reflected in the decoration mode.

Why NiO

IO is mainly oriented to stream data. In order to process individual bytes or characters, it is necessary to perform several object level method calls. This object-oriented processing method combines different IO objects and provides a high degree of flexibility (decoration mode in IO). However, when a large amount of data is needed, it may cause fatal damage to the execution efficiency. The ultimate goal of IO is efficiency, and efficient IO often fails to form a one-to-one correspondence with objects. Efficient IO often means that you have to choose the shortest path from a to B, and when performing a large number of IO operations, the complexity destroys the execution efficiency. The IO abstraction on the traditional Java platform works well and can be used widely. However, when a large amount of data is moved, these IO classes are not scalable, and do not provide the common IO functions that most operating systems have today, such as file locking, non block IO, readiness selection and memory mapping. These features are very important to achieve scalability, and also essential to maintain normal interaction with non Java applications, especially at the enterprise application level. However, the traditional Java IO mechanism does not simulate these general IO services. Java specification request ා 51 (JSR 51, https://jcp.org/en/jsr/detail?id=51 )It includes a detailed description of the high-speed and scalable I / O characteristics, which can make the IO performance of the underlying operating system better. The result of the implementation of JSR 51 is that the new classes are combined to form a java.nio And its sub package java.util.regex At the same time, some modifications have been made to the existing software package. The JCP website introduces the operation process of JSR in detail, as well as the evolution of NiO from the initial proposal to the final implementation and release. With the release of Merlin (jdk1.4), the powerful IO features of the operating system can be brought into full play with the tools provided by Java. When it comes to IO performance, Java is no worse than any other programming language.

At this point, we know that the purpose of Java NiO is to improve efficiency and make full use of the IO features provided by the operating system, so in order to deal with more processing requests, we need a new IO model (NiO).

NIOCore components of

In this section, we’ll start with NiO.

As mentioned above, NiO has three core components: selector, channel and buffer. Use a graph to abstract the relationship between the three.

 

 

Before Java NiO, the traditional IO usually used thread per to process network connection Task, that is, one thread one connection mode, can basically meet the requirements in small and medium-sized business processing. However, with the increasing number of connections, the created threads will continue to occupy memory space. At the same time, a large number of threads will also bring frequent context switching. CPU is used to operate context switching, which will inevitably affect the actual business processing. With Java After NiO, a small number of threads can be used to process a large number of connections. In the above figure, selector is the package of select / poll / epoll under Linux, and channel is the packaging of hardware, files, sockets and other program components that can be operated by io. We can regard channel as a network connection. There are mainly events such as connect, accept, read and write, and the selector is responsible for monitoring Once an event occurs on a channel, thread switches to the channel for event processing. Similar to the operating system level, if the select / poll mode is used, the application process scans the file descriptors of each socket (channel) in order to check whether they are ready and block on the select (selector). If they are ready, recvfrom is called. If epoll mode is used, it is not sequential scanning, but provides callback functions. When the file descriptor is ready, the callback function is called directly To further improve efficiency. Another component in the figure is buffer, which is actually a piece of memory. The bottom layer is implemented based on array. Generally, it appears in pairs with channel. Data reading and writing are realized through buffer.

Next, take a look at selector, channel, and buffer in turn.

Selectormodular

Selector

Selector is a multiplexed selectablechannel object.

Selectors can be created by calling their own open method. In the open method, selectors are created through the default selector provider of the system. Of course, you can also customize a selector by calling openselector. A selector will remain open until the close method is called.

The behavior of an optional channel object registered with the selector is represented by the selectionkey object. The selector maintains a collection of three selectionkeys:

The key set contains all the keys corresponding to the current channel registered with the selector. These keys can be returned through keys();

Each member of the selected key set is the relevant channel selected (in the previous select operation)

Operations that are determined to be ready and included in the interest collection of the key. This collection is returned through the selectedkeys () method. The selected key set is a subset of key set;

The cancelled key set is a collection of keys that have been cancelled but not yet unregistered. This set cannot be accessed directly. The cancelled key set is also a subset of key set.

For a newly created selector, the above three collections are empty by default.

By calling the register method of the channel, a new key will be added to the key set of the selector. During the selection operation, the cancelled keys will be removed from the key set, and the key set itself cannot be modified directly.

Whether the channel is closed directly or the close method of selectionkey is called, a key will be added to the cancelled key set. In the next selection operation, canceling a key will cause the corresponding channel to be unregistered, and the key will also be removed from the key set of the selector.

When performing the selection operation, keys will be added to the selected key set. Through the remove method of set or the remove method of iterator, the key can be directly removed from the selected key set. No other method can achieve this effect. In particular, removal is not a side effect of selection. Key cannot be directly added to the selected key set.

Selection

In each selection operation, keys may be added or deleted from selected key set, key set or cancelled key set. The selection operation is performed through the select(), select (long) and selectnow() methods. Generally, it includes the following three steps:

1. Each key in the cancelled key set can be removed from the key set to which it belongs. At the same time, the channel to which it belongs will also be unregistered. In this step, the cancelled key set will be empty;

2. The underlying operating system starts to be queried to update the ready status of the remaining channel channels to execute the events of interest to the key. For a channel with at least one such operation, the following two actions will be executed:

2.1 if the channel key is not in the selected key set, then the key will be added to the selected key set, and its ready operation will be modified to mark accurately which channel has completed the preparation work, and the ready information of any previous ready set will be discarded;

2.2 if the channel key is in the selected key set, and its ready operation will be modified to accurately mark which channel has completed the preparation, the ready information of any previous ready set will be retained. In other words, the ready set of the underlying operating system will be written to the current ready set of the key bit by bit.

If there is no interest set in all key sets at the beginning, neither the selected key set nor its corresponding ready operation will be updated.

3. If a new key is added to the cancelled key when step 2 is executed, proceed with step 1.

The essential difference between these three methods is to select whether the operation is blocked, wait for one or more channels to be ready, or how long to wait.

Concurrency

The selector itself can be safely used by multiple concurrent threads. However, their key set is not.

During the selection operation, the selector is synchronized on the selector object, followed by key set and finally selected key set, in this order. The cancelled key set is also synchronized between steps 1 and 3 of the selection process.

Changes to the selector’s interest sets have no effect on the selection operation when it is made, and they will see it in the next selection operation.

The presence of a key in one or more keysets of a selector does not indicate that the key is valid or its channels are open. If it is possible for other threads to cancel keys or close channels, application code should synchronize carefully and check these conditions if necessary.

Threads will block on the select() or select (long) method. If other threads want to interrupt the blocking, they can do so in the following three ways:

By calling the wakeup method of the selector;

By calling the close method of the selector;

By calling the interrupt method of the blocked thread, in this case, its interrupt state is set and the wakeup method of the selector is called.

The close method synchronizes on the selector and all three keysets in the same order as the select operation.

In general, the selector key and selected key cannot be safely used by multiple concurrent threads. If such threads can directly modify one of these collections, access should be controlled by synchronizing on the collection itself. The iterator methods of these collections return iterators that fail quickly: if the collection is modified after the iterator is created, it is thrown in any way other than by calling the iterator’s own remove method java.util.ConcurrentModificationException 。

All the methods provided for the selector are as follows:

 

 

SelectionKey

Represents the token that the selectablechannel registers with the selector.

Each time a channel is registered in the selector, a selection key is created. This key remains in effect until it is cancelled by calling its cancel method, closing its channel or closing its selector. Instead of immediately removing it from the selector, the cancel key adds it to the selector’s cancel keys collection to remove it during the next selection operation. You can test the validity of the key by calling the isvalid method.

The selection key contains two sets of operations represented as integer values. Each bit of the operation set represents the class of optional operations supported by the channel of the key.

The interest set determines one of the selection methods for the next invocation of the selector, and tests the readiness of those action categories. The interest set is initialized with the value given when the key was created, which can be changed later through the interestsops (int) method.

The prepared set identifies the type of operation for which the key’s channel is ready. When a key is created, the preparation set is initialized to zero. It may be updated later by the selector during the select operation, but it cannot be updated directly.

The ready set of a selection key indicates that its channel has prompted for an operation class, but it is not a guarantee that operations in such a class can be performed by threads without causing thread blocking. The preparation is likely to be accurate immediately after the selection operation is completed. External events and I / O operations called on the corresponding channels can make it inaccurate.

This class defines all known operation set bits, but the exact bit supported by a given channel depends on the type of channel. Each subclass of selectablechannel defines a validops () method that returns a collection that identifies only those operations that the channel supports. Attempting to set or test an operation set that is not supported by a key channel will result in a run-time exception.

It is often necessary to associate some application specific data with a selection key, for example, an object represents the state of a higher-level protocol and processes ready notifications to implement that protocol. Therefore, the selection key supports attaching a single arbitrary object to the key. You can attach objects through the attach method, and then retrieve them through the attach method.

The selection key can be safely used for multiple concurrent threads. In general, the operations of reading and writing interest sets are synchronized with some operations of the selector. Specifically, how this synchronization is performed depends on the implementation: in a low-performance implementation, if the select operation is already in progress, the read and write of the interest group may be blocked indefinitely; in a high-performance implementation, the read or write interest set may be temporarily blocked, if any. In any case, the selection action always uses the current interest setting value at the beginning of the operation.

All the methods provided for selectionkey are as follows:

 

 

 

Channelmodular

Channel is used to represent an open connection to an entity such as a hardware device, file, network socket, or program component that can perform one or more different I / O operations (such as read or write). I / O can be divided into two broad categories: file I / O and stream I / O. Accordingly, there are two types of channels, file channel and socket channel. File channels have one filechannel class, while sockets have three socket channel classes: socketchannel, serversocketchannel, and datagramchannel. Channels can operate in blocking or nonblocking mode. A non blocking channel never sleeps the calling thread. The requested operation either completes immediately or returns a result indicating that no action was taken. Only stream oriented channels such as socketchannel and serversocketchannel can use non blocking mode. Socketchannel and serversocketchannel are derived from selectable channel. Classes derived from selectablechannel can be used with selectors that support ready selectio. By combining nonblocking I / O and selectors, you can use multiplexed I / O, which is select / poll / epoll. Since filechannel is not extended from the selectablechannel class, filechannel, that is, file IO, cannot use the non blocking model.

 

FileChannel

Channels for reading, writing, mapping, and manipulating files.

A file channel is a seekablebytechannel that can be connected to a file. It has the current position in the file and supports query and modification. The file itself contains a variable length sequence of bytes, which can be read and written, and its current size can be queried. When the write bytes exceed the current size, the file size increases; the file size decreases when truncated. The file may also have associated metadata, such as access rights, content type, and last modified time, which does not define methods for metadata access.

In addition to the familiar byte read, write, and close operations, this class defines the following file specific operations:

The byte can be read or written in the absolute position of the file without affecting the current position of the channel;

The area of a file can be mapped directly to memory. For large files, this is usually much more efficient than calling traditional read or write methods;

Updates to files may be forced to the underlying storage device to ensure that data is not lost in the event of a system crash;

Bytes can be transferred from the file to other channels, and vice versa. It can be optimized by the operating system to transfer the bytes to or from the file system cache;

The area of the file may be locked to prevent access by other programs;

File channels can be safely used by multiple concurrent threads. As specified by the channel interface, the close method can be called at any time. At any given time, there may be only one operation involving the location of the channel or the size of its file that can be changed. When the first such operation is still in progress, attempts to start a second such operation are blocked until the first operation completes. Other operations, especially those taking a clear stand, can be carried out at the same time. Whether they are actually executed depends on the underlying implementation.

Instances of this class provide file views that are consistent with other views of the same file provided by other instances of the same program. However, due to the latency caused by caching and network file system protocols executed by the underlying operating system, the views provided by such instances may or may not be consistent with those seen by other concurrent programs. This is true regardless of the language in which these other programs are written and whether they are running on the same computer or on another computer. The exact nature of any such inconsistency depends on how the underlying operating system implements it.

Create a file channel by calling the open method defined by this class. You can also get file channels from existing FileInputStream, fileoutputstream, or RandomAccessFile objects by calling the getchannel method of subsequent classes, which returns the file channels connected to the same underlying file. If a file channel is obtained from an existing stream or a random access file, the state of the file channel is closely related to the state of the object that the getchannel method returns to the channel. Whether you change the channel location explicitly, or by reading or writing bytes, you change the file location of the original object, and vice versa. Changing the length of a file through a file channel changes the length seen through the original object, and vice versa. Changing the contents of a file by writing bytes changes what the original object sees, and vice versa.

At each point, this class specifies that an instance of “readable,” “writable,” or “readable and writable” is required. The channel obtained through the getchannel method of the FileInputStream instance is opened for reading. The channel obtained through the getchannel method of the fileoutputstream instance is opened for writing. Finally, if the instance was created with the pattern “R”, the channel obtained through the getchannel method of the RandomAccessFile instance will be opened for reading; if the instance was created with the pattern “RW”, it will be opened for reading and writing. The file channel opened for writing may be in attached mode, for example, if it is obtained from a file output stream created by calling the fileoutputstream (file, Boolean) constructor and passing true for the second parameter. In this mode, each call to a relative write operation advances the position to the end of the file before writing the requested data. Whether the position promotion and data writing are completed in a single atomic operation depends on the specific implementation of the operating system.

SocketChannel

An optional channel for stream oriented connection socket.

Create a socket channel by calling the open method of this class. Cannot create a channel for any existing socket. As soon as the new socket channel is opened, it is not connected. A connection that was not raised with a connectedo channel will result in an operation that was not called up with a connectedo channel. The socket channel can connect by calling its connect method. After connecting, the socket channel will remain connected until it is closed. Whether the socket channel is connected can be determined by calling its isconnected method.

Socket channel supports non blocking connection. A socket channel can be created, and the process of establishing a link to a remote socket can be started by the connect method, and then completed by the finishconnect method. You can call the isconnectionpending method to determine if the connection operation is in progress.

Socket channels support asynchronous close, similar to the asynchronous close operation specified in the channel class. If the input of a socket is closed by one thread and another thread is blocked during a read operation of the socket channel, the read operation in the blocking thread will complete without reading any bytes, and will return – 1. If the output of a socket is closed by one thread and another thread is blocked during a write operation on the socket channel, the blocked thread will receive an asynchronous closeexception.

Socket options are configured using the setoption method. Socket channels support the following options:

Option name Description

SO_ Size of sndbuf socket send buffer

SO_ Size of rcvbuf socket receive buffer

SO_ Keepalive keeps the connection active

SO_ Reuseaddr reuse address

SO_ Linger on shutdown if data is available (configured only in blocking mode)

TCP_ Nodelay disable Nagle algorithm

Other (implementation specific) options can also be supported.

Socket channels can be safely used by multiple concurrent threads. They support concurrent reads and writes, although at most one thread can be read at any given time and up to one thread can be written. The connect and finishconnect methods are synchronized with each other, and attempts to start a read or write operation while a call to one of these methods is in progress is blocked until the call completes.

 

ServerSocketChannel

An optional channel for stream oriented listening socket.

The server socket channel can be created by calling the open method of this class. Cannot create a channel for any existing ServerSocket. The newly created server socket channel is unbound as soon as it is opened. Attempting to call the accept method of an unbound server socket channel causes a notyetboundexception to be thrown. You can bind a server socket channel by calling one of the bind methods defined by this class.

Socket options are configured using the setoption method. The server socket channel supports the following options:

Option name Description

SO_ Size of rcvbuf socket receive buffer

SO_ Reuseaddr reuse address

Other (implementation specific) options can also be supported.

The server socket channel can be safely used for multiple concurrent threads.

 

Buffermodular

Buffer

A container for data of a particular primitive type.

A buffer is a linear finite sequence of elements of a particular primitive type. In addition to its content, the basic properties of the buffer include capacity, limit, and position

The capacity of a buffer is the number of elements it contains. The buffer capacity is never negative and does not change.

The limit of the buffer is the index of the first element that should not be read or written. The limit of the buffer is never negative and never greater than the capacity of the buffer.

The position of the buffer is the index of the next element to read or write. The position of the buffer will never be negative or greater than the limit.

For each non Boolean primitive type, this class has a subclass, namely intbuffer, shortbuffer, longbuffer, charbuffer, ByteBuffer, doublebuffer, and floatbuffer.

 

Transferring data

Each subclass of this class defines two classes of get and put operations:

A relative operation reads or writes one or more elements from the current location, and then increases the number of elements transferred at that location. If the requested transfer exceeds the limit, the relative get operation raises a bufferunderflowexception and the relative put operation raises a bufferoverflow exception; in either case, the data is not transferred.

The absolute operation uses explicit element index and does not affect the position. If the index parameter exceeds the limit, absolute get and put operations raise indexoutofboundsexception.

Of course, data can also be moved into or out of the buffer by I / O operations that are always relative to the current location of the channel.

Marking and resetting

The mark mark mark of the buffer is the index whose position is reset when the reset method is called. Mark is not always defined, but when defined, it is never negative and never greater than position. If a mark is defined, the mark mark is discarded when the position or limit is adjusted to a value less than mark. If mark is not defined, calling the reset method raises an invalidmarkexception.

Invariants

For mark, position, limit and capacity, the following invariants hold:

0 <=mark<= position <=limit<=capacity

Newly created buffers always have zero positions and undefined tags. Initially, limit can be zero or some other value, depending on the type of buffer and how it is constructed. Each element of the newly allocated buffer is initialized to zero.

Clearing, flipping, and rewinding

In addition to the methods for accessing position, limit, capacity, mark and reset, this class also defines the following operations on the buffer:

Clear prepares the buffer for a new channel read or relative put operation sequence: set limit to capacity and position position to zero.

Flip prepares the buffer for a new channel write or relative get operation sequence: set limit to current position, and then position to zero.

Rewind prepares the buffer to reread the data it already contains: leave the limit unchanged and set position to zero.

Read-only buffers

Each buffer is readable, but not every buffer is writable. The mutation method for each buffer class is specified as an optional operation, which raises a readonlybufferexception when called on a read-only buffer. Read only buffers are not allowed to change their contents, but their mark, positoin and limit are variable. Whether the buffer is read-only can be determined by calling the isReadOnly method.

Thread safety

Buffers cannot be safely used for multiple concurrent threads. If a buffer is to be used by multiple threads, access to the buffer should be controlled by appropriate synchronization.

Invocation chaining

No other method in this class to return values is specified to return the buffer on which they were called. This allows method calls to be linked together, for example, a sequence of statements:

    b.flip();

    b.position(23);

    b.limit(42);

It can be replaced by a more compact statement

    b.flip().position(23).limit(42);

A simple chat program based on NiO

After summarizing the basic knowledge of NiO, we know that NiO can handle file IO and streaming IO (network IO). The greatest charm of NiO lies in the processing of network io. Next, we will implement a simple chat program through NiO to continue to understand java NiO. This simple chat program is a server with multiple clients, and the clients can realize data communication with each other.

Server:

public class NioServer {
    //Use map to record client connection information
    private static Map clientMap = new HashMap();

    public static void main(String[] args) throws Exception {
        //Create a serversocketchannel to listen to the port
        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
        //Configured as non blocking
        serverSocketChannel.configureBlocking(false);
        //Get server socket
        ServerSocket serverSocket = serverSocketChannel.socket();
        //Monitoring port 8899
        serverSocket.bind(new InetSocketAddress(8899));
        //Create selector
        Selector selector = Selector.open();
        //When the serversocketchannel registers with the selector, it pays attention to the connection events of the client
        serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

        while (true) {
            try {
                //Blocking attention to events of interest
                selector.select();
                //Gets the selectionkey collection of the concerned events
                Set selectionKeys = selector.selectedKeys();
                //Do different processing according to different events
                selectionKeys.forEach(selectionKey -> {
                    final SocketChannel client;
                    try {
                        //After the connection is established, it starts to listen for the read and write events of the client
                        if (selectionKey.isAcceptable()) {
                            //How to listen to the client read / write events, the client connection needs to be registered with the selector first
                            //How to obtain the channel established by the client can be accessed through selectionKey.channel ()
                            //Only serversocketchannel is registered, so the channel entering this branch must be serversocketchannel
                            ServerSocketChannel server = (ServerSocketChannel)selectionKey.channel();
                            //Get the real client
                            client = server.accept();
                            client.configureBlocking(false);
                            //Client connection registered with selector
                            client.register(selector,SelectionKey.OP_READ);
                            //Selectors are registered with serversocketchannel and socketchannel
                            //UUID stands for client ID, here is business information
                            String key = "[" + UUID.randomUUID().toString() + "]";
                            clientMap.put(key,client);
                        }else if (selectionKey.isReadable()) {
                            //Processing the data written by the client is readable data for the server, which must be socketchannel
                            client = (SocketChannel)selectionKey.channel();
                            //Channel cannot read and write data, and must read and write data through buffer
                            ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
                            //The server reads data to buffer
                            int count = client.read(byteBuffer);
                            if(count > 0) {
                                //Read write conversion
                                byteBuffer.flip();
                                //Write data to other clients
                                Charset charset = Charset.forName("utf-8");
                                String receiveMessage = String.valueOf(charset.decode(byteBuffer).array());

                                System.out.println("client:" + client + receiveMessage);

                                String sendKey = null;
                                for(Map.Entry entry: clientMap.entrySet()) {
                                    if(client == entry.getValue()) {
                                        //Get the sender's UUID to simulate the client's chat sending information
                                        sendKey = entry.getKey();
                                        break;
                                    }
                                }
                                //Send information to all clients
                                for(Map.Entry entry: clientMap.entrySet()) {
                                    //Get all the client objects that establish the connection
                                    SocketChannel value = entry.getValue();

                                    ByteBuffer writeBuffer = ByteBuffer.allocate(1024);
                                    //This put operation is a read operation of buffer
                                    writeBuffer.put((sendKey + ":" + receiveMessage).getBytes());
                                    //Read write conversion is required before write
                                    writeBuffer.flip();
                                    //Write it out
                                    value.write(writeBuffer);
                                }

                            }
                        }
                    }catch (Exception ex) {
                        ex.printStackTrace();
                    }
                });
                //After processing, the key must be deleted, otherwise it will be repeatedly processed and an error will be reported
                selectionKeys.clear();
            }catch (Exception e) {
                e.printStackTrace();
            }
        }
    }
}

 

client:

public class NioClient {


    public static void main(String[] args) throws Exception {
        //Create socketchannel to request port
        SocketChannel  socketChannel = SocketChannel.open();
        //Configured as non blocking
        socketChannel.configureBlocking(false);
        //Create selector
        Selector selector = Selector.open();
        //When the socket channel registers with the selector, it pays attention to the event of establishing connection to the server
        socketChannel.register(selector,SelectionKey.OP_CONNECT);
        //Initiate connection to remote
        socketChannel.connect(new InetSocketAddress("127.0.0.1",8899));

        while (true) {
            //Blocking attention to events of interest
            selector.select();
            //Gets the selectionkey collection of the concerned events
            Set selectionKeys = selector.selectedKeys();
            //Do different processing according to different events
            for(SelectionKey selectionKey : selectionKeys) {
                final SocketChannel channel;
                if(selectionKey.isConnectable()) {
                    //Establish a good connection with the server to obtain the channel
                    channel = (SocketChannel)selectionKey.channel();
                    //Is the client and server in connection
                    if(channel.isConnectionPending()) {
                        //Complete the connection
                        channel.finishConnect();
                        //Send connection establishment information
                        ByteBuffer writeBuffer = ByteBuffer.allocate(1024);
                        //Read in
                        writeBuffer.put (( LocalDateTime.now () + "connected successfully"). Getbytes());
                        writeBuffer.flip();
                        //Write
                        channel.write(writeBuffer);
                        //Establishment of TCP two way channel
                        //Keyboard as standard input to avoid the main thread blocking, the new thread to do processing
                        ExecutorService service = Executors.newSingleThreadExecutor(Executors.defaultThreadFactory());
                        service.submit(() -> {
                           while (true) {
                               writeBuffer.clear();
                               //IO operation
                               InputStreamReader inputStreamReader = new InputStreamReader(System.in);
                               BufferedReader reader = new BufferedReader(inputStreamReader);

                               String readLine = reader.readLine();
                               //Read in
                               writeBuffer.put(readLine.getBytes());
                               writeBuffer.flip();
                               //Write
                               channel.write(writeBuffer);
                            }
                        });
                    }
                    //The client also needs to listen to the server for writing information, so it needs to pay attention to the read event
                    channel.register(selector,SelectionKey.OP_READ);
                }else if(selectionKey.isReadable()) {
                    //Reading events from the server
                    channel = (SocketChannel)selectionKey.channel();
                    ByteBuffer readBuffer = ByteBuffer.allocate(1024);

                    int count = channel.read(readBuffer);
                    if(count > 0) {
                        readBuffer.flip();

                        Charset charset = Charset.forName("utf-8");
                        String receiveMessage = String.valueOf(charset.decode(readBuffer).array());

                        System.out.println("client:" + receiveMessage);
                    }
                }
                //After processing, the key must be deleted, otherwise it will be repeatedly processed and an error will be reported
                selectionKeys.clear();
            }

        }


    }
}

Demonstration effect:

 

 

 

 

 

 

 

 

 

 

 

 

Finally, let’s summarize:

1. IO is flow oriented, NiO is buffer oriented, and flow can only be transmitted in one direction, while buffer can be transmitted in two directions. In addition to increasing throughput, this model is closer to the bottom layer of operating system and network;

2. For network IO, selector and channel are combined to realize IO multiplexing. In this way, a small number of threads can also handle a large number of connections, which is suitable for high concurrency and large traffic scenarios. For file IO, IO multiplexing is not mentioned. However, filechannel can greatly improve file IO by providing transferto and transferfrom methods to reduce the number of underlying copies Performance;

3. Buffer buffer is used to store data. Except that there is no boolean type, other basic data types are the same as those in Java. The core attributes of buffer are position, limit and capacity. These variables are constantly flipped and changed when reading and writing data. However, the design is not elegant. ByteBuffer of netty provides a way to separate read and write indexes to make the implementation more convenient Add elegance;

4. NiO programming mode summary:

Register the socket channel into the selector to listen for the events of interest;

When the interested event is ready, it will be processed in the method we handle;

Every time the ready event is processed, the selection key is deleted (because we have finished processing it).

 

 

reference material:

Java IO教程

Java NIO 系列教程

https://segmentfault.com/a/1190000014932357?utm_source=tag-newest

https://www.zhihu.com/question/29005375?sort=created

Some of the pictures are from a learning video. Please let me know if there is any infringement.

Recommended Today

Application of Linux page cache tuning in Kafka

This article starts with WeChat official account of vivo Internet technology.Link:https://mp.weixin.qq.com/s/MaeXn-kmgLUah78brglFkg Author: Yang Yijun This paper mainly describes the background of Linux page cache optimization, the basic concept of page cache, lists some solutions for the IO performance bottleneck of Kafka, how to adjust the relevant parameters of page cache, and the effect comparison before […]