Buffer#

In traditional BIO, data I/O is stream-oriented, divided into byte streams and character streams.

In Java 1.4’s NIO library, all data is handled with buffers. This is an important difference from the previous BIO, somewhat similar to buffered streams in BIO. NIO reads data directly into the Buffer during read operations and writes data into the Buffer during write operations. When using NIO, reading and writing data is always done through buffers.

Among the subclasses of Buffer, the most commonly used is ByteBuffer, which can store and operate on byte data.

You can think of a Buffer as an array; IntBuffer, FloatBuffer, CharBuffer, etc., correspond to int[], float[], char[], etc.

To better understand buffers, let’s briefly look at the four member variables defined in the Buffer class:

1
public abstract class Buffer {
2
    // Invariants: mark <= position <= limit <= capacity
3
    private int mark = -1;
4
    private int position = 0;
5
    private int limit;
6
    private int capacity;
7
}

The specific meanings of these four member variables are as follows:

1. Capacity: The maximum amount of data the Buffer can store; set when the Buffer is created and cannot be changed.
2. Limit: The boundary for reading/writing data in the Buffer. In write mode, limit represents the maximum writable data, usually equal to capacity (can be set via limit(int newLimit)); in read mode, limit equals the amount of data actually written into the Buffer.
3. Position: The index of the next element to be read or written. When switching from write mode to read mode (flip), position is reset to zero so you can read/write from the start.
4. Mark: Buffer allows you to set the position to a marked point; this is optional.

And these variables satisfy the relation: 0 <= mark <= position <= limit <= capacity.

Additionally, Buffer has two modes: read mode and write mode, used to read data from the Buffer or write data into the Buffer. After a Buffer is created, it defaults to write mode. Calling flip() switches to read mode. To switch back to write mode, you can call clear() or compact().

Buffer objects cannot be created with new; they must be instantiated via static methods.

Here we use ByteBuffer as an example:

1
// Allocate heap memory
2
public static ByteBuffer allocate(int capacity);
3
// Allocate direct memory
4
public static ByteBuffer allocateDirect(int capacity);

Buffer’s two most core methods:

1. get: reads data from the buffer
2. put: writes data into the buffer

Besides these two methods, other important methods:

• flip: switches the buffer from write mode to read mode; it sets limit to the current position value and sets position to 0.
• clear: clears the buffer, switches from read mode to write mode, and sets position to 0 and limit to capacity.
• ……

Buffer data flow:

1
import java.nio.*;
2

3
public class CharBufferDemo {
4
    public static void main(String[] args) {
5
        // Allocate a CharBuffer with capacity 8
6
        CharBuffer buffer = CharBuffer.allocate(8);
7
        System.out.println("Initial state:");
8
        printState(buffer);
9

10
        // Write 3 characters to the buffer
11
        buffer.put('a').put('b').put('c');
12
        System.out.println("State after writing 3 characters:");
13
        printState(buffer);
14

15
        // Call flip() to prepare reading: position becomes 0, limit becomes 3
16
        buffer.flip();
17
        System.out.println("State after calling flip():");
18
        printState(buffer);
19

20
        // Read characters
21
        while (buffer.hasRemaining()) {
22
            System.out.print(buffer.get());
23
        }
24

25
        // Call clear() to clear the buffer: position becomes 0, limit becomes capacity
26
        buffer.clear();
27
        System.out.println("State after calling clear():");
28
        printState(buffer);
29

30
    }
31

32
    // Print capacity, limit, position, and mark of the buffer
33
    private static void printState(CharBuffer buffer) {
34
        System.out.print("capacity: " + buffer.capacity());
35
        System.out.print(", limit: " + buffer.limit());
36
        System.out.print(", position: " + buffer.position());
37
        System.out.print(", mark 开始读取的字符: " + buffer.mark());
38
        System.out.println("\\n");
39
    }
40
}

Output:

1
初始状态：
2
capacity: 8, limit: 8, position: 0
3

4
写入3个字符后的状态：
5
capacity: 8, limit: 8, position: 3
6

7
准备读取buffer中的数据！
8

9
调用flip()方法后的状态：
10
capacity: 8, limit: 3, position: 0
11

12
读取到的数据：abc
13

14
调用clear()方法后的状态：
15
capacity: 8, limit: 8, position: 0

To make this easier to understand, I drew a diagram showing how capacity, limit, and position change at each stage.

Channel#

A Channel is a conduit that establishes a connection with a data source (such as a file, network socket, etc.). We can use it to read and write data, as if opening a water pipe through which data flows freely in the Channel.

In BIO, streams are unidirectional, categorized as various InputStream (input streams) and OutputStream (output streams); data only transfers in one direction. The difference between channels and streams is that channels are bidirectional and can be used for reading, writing, or both simultaneously.

Channel interacts with the Buffers as introduced above: during read, data from the Channel is filled into the Buffer, and during write, data from the Buffer is written into the Channel.

Additionally, because a Channel is full-duplex, it can map better to the underlying OS APIs. In particular, in the UNIX network programming model, the OS channels are full-duplex and support both read and write operations.

Among the subclasses of Channel, the most commonly used are the following types:

• FileChannel: file access channel;
• SocketChannel, ServerSocketChannel: TCP communication channels;
• DatagramChannel: UDP communication channel;

Two core methods of Channel:

1. read: reads data and writes it into a Buffer.
2. write: writes data from the Buffer into the Channel.

Here we use FileChannel as an example to demonstrate reading file data.

1
RandomAccessFile reader = new RandomAccessFile("test_read.in", "r"))
2
FileChannel channel = reader.getChannel();
3
ByteBuffer buffer = ByteBuffer.allocate(1024);
4
channel.read(buffer);

Selector#

Selector is a key component in NIO that allows a single thread to handle multiple Channels. Selector is based on an event-driven I/O multiplexing model, with the main operating principle: by registering channel events with the Selector, the Selector will continuously poll the Channels registered on it.

When events occur, such as a Channel having a new TCP connection, or read and write events, the Channel becomes ready and will be selected by the Selector. The Selector will add the relevant Channels to the ready set. Through SelectionKey you can obtain the set of ready Channels and perform the corresponding I/O operations on these ready Channels.

A multiplexer, Selector, can poll multiple Channels simultaneously. Because the JDK uses epoll() instead of the traditional select implementation, there is no fixed limit like maximum connections 1024/2048. This means a single thread can handle the Selector’s polling and connect thousands of clients.

Selector can monitor the following four event types:

1. SelectionKey.OP_ACCEPT: indicates a channel accepting connections; this is typically used with ServerSocketChannel.
2. SelectionKey.OP_CONNECT: indicates a channel has completed connection; typically used with SocketChannel.
3. SelectionKey.OP_READ: indicates a channel is ready for reading, i.e., there is data to read.
4. SelectionKey.OP_WRITE: indicates a channel is ready for writing, i.e., data can be written.

Selector is an abstract class and can be instantiated by calling its static method open(). A Selector can monitor the IO status of multiple SelectableChannels and is the core of non-blocking IO.

A Selector instance has three SelectionKey collections:

1. All SelectionKey collection: Represents the Channels registered on this Selector; this set can be obtained via the keys() method.
2. Selected SelectionKey collection: Represents all Channels available via the select() method that require IO processing; this set can be returned by selectedKeys().
3. Cancelled SelectionKey collection: Represents all Channels whose registration has been cancelled; on the next invocation of select(), the corresponding SelectionKeys will be removed. Programs usually do not access this collection directly, and there is no exposed accessor.

A simple demonstration of iterating over the selected SelectionKey set and handling it:

1
Set<SelectionKey> selectedKeys = selector.selectedKeys();
2
Iterator<SelectionKey> keyIterator = selectedKeys.iterator();
3
while (keyIterator.hasNext()) {
4
    SelectionKey key = keyIterator.next();
5
    if (key != null) {
6
        if (key.isAcceptable()) {
7
            // ServerSocketChannel accepted a new connection
8
        } else if (key.isConnectable()) {
9
            // A new connection established
10
        } else if (key.isReadable()) {
11
            // Channel has data ready to read
12
        } else if (key.isWritable()) {
13
            // Channel is ready for writing
14
        }
15
    }
16
    keyIterator.remove();
17
}

Selector also provides a series of methods related to select():

• int select(): Monitors all registered Channels; when any require IO processing, this method returns and the corresponding SelectionKeys are added to the selected set; this method returns the number of Channels.
• int select(long timeout): A select() operation with a timeout.
• int selectNow(): Performs an immediate-return select() operation; unlike the parameterless select(), this method does not block.
• Selector wakeup(): Causes a select() that is currently blocking to return immediately.
• …

Using Selector to implement network I/O:

1
import java.io.IOException;
2
import java.net.InetSocketAddress;
3
import java.nio.ByteBuffer;
4
import java.nio.channels.SelectionKey;
5
import java.nio.channels.Selector;
6
import java.nio.channels.ServerSocketChannel;
7
import java.nio.channels.SocketChannel;
8
import java.util.Iterator;
9
import java.util.Set;
10

11
public class NioSelectorExample {
12

13
  public static void main(String[] args) {
14
    try {
15
      ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
16
      serverSocketChannel.configureBlocking(false);
17
      serverSocketChannel.socket().bind(new InetSocketAddress(8080));
18

19
      Selector selector = Selector.open();
20
      // Register the ServerSocketChannel with the Selector to listen for OP_ACCEPT
21
      serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
22

23
      while (true) {
24
        int readyChannels = selector.select();
25

26
        if (readyChannels == 0) {
27
          continue;
28
        }
29

30
        Set<SelectionKey> selectedKeys = selector.selectedKeys();
31
        Iterator<SelectionKey> keyIterator = selectedKeys.iterator();
32

33
        while (keyIterator.hasNext()) {
34
          SelectionKey key = keyIterator.next();
35

36
          if (key.isAcceptable()) {
37
            // Handle connection event
38
            ServerSocketChannel server = (ServerSocketChannel) key.channel();
39
            SocketChannel client = server.accept();
40
            client.configureBlocking(false);
41

42
            // Register client channel with the Selector to listen for OP_READ
43
            client.register(selector, SelectionKey.OP_READ);
44
          } else if (key.isReadable()) {
45
            // Handle read event
46
            SocketChannel client = (SocketChannel) key.channel();
47
            ByteBuffer buffer = ByteBuffer.allocate(1024);
48
            int bytesRead = client.read(buffer);
49

50
            if (bytesRead > 0) {
51
              buffer.flip();
52
              System.out.println("收到数据：" +new String(buffer.array(), 0, bytesRead));
53
              // Register client channel with the Selector to listen for OP_WRITE
54
              client.register(selector, SelectionKey.OP_WRITE);
55
            } else if (bytesRead < 0) {
56
              // Client disconnected
57
              client.close();
58
            }
59
          } else if (key.isWritable()) {
60
            // Handle write event
61
            SocketChannel client = (SocketChannel) key.channel();
62
            ByteBuffer buffer = ByteBuffer.wrap("Hello, Client!".getBytes());
63
            client.write(buffer);
64

65
            // Register client channel with the Selector to listen for OP_READ
66
            client.register(selector, SelectionKey.OP_READ);
67
          }
68

69
          keyIterator.remove();
70
        }
71
      }
72
    } catch (IOException e) {
73
      e.printStackTrace();
74
    }
75
  }
76
}

In the example, we create a simple server that listens on port 8080 and uses a Selector to handle connection, read, and write events. When data is received from a client, the server reads the data and prints it to the console, then replies to the client with “Hello, Client!”.