套接字文件描述符消耗小细节分析

 

      套接字是通信端点的抽象。正如使用文件描述符访问文件，应用程序用套接字描述符访问套接字。套接字描述符在UNIX系统中被当作是一种文件描述符。事实上，许多处理文件描述符的函数（read和write）可以用于处理套接字描述符。 ——《unix环境高级编程》

 

      WEB应用客户端（浏览器或APP等）与服务端的通信大多数是通过HTTP或HTTPS协议进行的。而套接字（Socket）应该算是OSI七层网络模型中应用层与传输层直接的抽象层，它是面向应用层的一个编程接口。而HTTP/HTTPS则位于应用层，就是通过套接字（Socket）进行信息通信的。所以，WEB应用的每个HTTP链接都与套接字（Socket）息息相关，特别是并发量大的应用，套接字（Socket）的使用至关重要。毕竟，它是应用出入的一个“口”。处理不当，则成为应用系统瓶颈。

 

      

      套接字（Socket）的瓶颈，大多数情况，还是套接字描述符的一个瓶颈（文件打开数过多）。正如开头所说：套接字是通信端点的抽象，应用对互联网的通讯就是通过对这个抽象进行读写，就像读写普通文件一样。在unix操作系统中，读写文件，就需要系统分配文件标识：文件描述符。套接字（Socket）也不例外。

 

      Talk is cheap,show code:

 

我在本地（window）用Java写了Socket服务端代码，并启动服务：

public class SocketServer{
public static void main(String[] args) {
try{
ServerSocket server=null;
server = new ServerSocket(8000);
Socket socket=null;
while(true){
socket=server.accept();
BufferedReader br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
PrintWriter pw = new PrintWriter(socket.getOutputStream());
System.out.println(br.readLine());
pw.println("ok");
pw.flush();
pw.close();
br.close();
}
}catch(Exception e){e.printStackTrace();}
}
}

 

另外也用Java写了一个Socket的客户端，用打包成jar放到测试机（SUSE Linux）下进行测试，并观察其对套接字描述符的一个消耗情况：

public class SocketClient{
public static void main(String[] args) throws Exception {
Socket socket=new Socket("192.168.22.15",8000);
Socket socket2=new Socket("192.168.22.15",8000);
BufferedReader br=new BufferedReader(new InputStreamReader(socket.getInputStream()));
BufferedReader br2=new BufferedReader(new InputStreamReader(socket2.getInputStream()));
PrintWriter pw = new PrintWriter(socket.getOutputStream());
PrintWriter pw2 = new PrintWriter(socket2.getOutputStream());
pw.println("hello");
pw2.println("hello2");
pw.flush();
pw2.flush();
System.out.println(br.readLine());
System.out.println(br2.readLine());
Thread.sleep(20000);
br.close();
br2.close();
pw.close();
pw2.close();
socket.close();
socket2.close();
System.out.println("socket close");
Thread.sleep(20000);
System.out.println("function close");
}
}

 

以上代码运行期间，我会通过lsof -p <SocketClient进程号>|wc –l命令观察此SocketClient进程消耗的文件描述符数量。

 

启动SocketClient.jar20秒前（OK）：

结果:101

启动SocketClient.jar20秒到40秒之间（socket close）：

结果:99

启动SocketClient.jar40秒后（function close）：

结果:0

 

启动SocketClient.jar20秒前lsof -p <SocketClient进程号>详情：

 

COMMAND  PID USER   FD   TYPE  DEVICE SIZE/OFF    NODE NAME

java    5920 root  cwd    DIR     8,3     4096  486764 /usr/local/test

java    5920 root  rtd    DIR     8,3     4096       2 /

java    5920 root  txt    REG     8,3    59141  878312 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/bin/java

java    5920 root  mem    REG     8,3   116348  878486 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/libnet.so

java    5920 root  mem    REG     8,3    88544   90416 /lib64/libgcc_s.so.1

java    5920 root  mem    REG     8,3    84914  878492 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/libzip.so

java    5920 root  mem    REG     8,3     9975  878465 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/libdbgwrapper.so

java    5920 root  mem    REG     8,3   108248   90308 /lib64/libnsl-2.11.3.so

java    5920 root  mem    REG     8,3   224723  878477 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/libjava.so

java    5920 root  mem    REG     8,3   140261  878396 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libj9ute24.so

java    5920 root  DEL    REG     8,3          2183413 /var/run/nscd/dbZxaX8m

java    5920 root  mem    REG     8,3    90320  878429 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libiverel24.so

java    5920 root  mem    REG     8,3   629488  878451 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libjclscar_24.so

java    5920 root  mem    REG     8,3   225264  878449 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9vrb24.so

java    5920 root  mem    REG     8,3   316148  878441 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9jvmti24.so

java    5920 root  mem    REG     8,3   187277  878433 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9dyn24.so

java    5920 root  mem    REG     8,3   841499  878434 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9gc24.so

java    5920 root  mem    REG     8,3   109715  878430 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9bcv24.so

java    5920 root  mem    REG     8,3  6235544  878438 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9jit24.so

java    5920 root  mem    REG     8,3    93053  878446 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9trc24.so

java    5920 root  mem    REG     8,3    74373  878399 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libj9zlib24.so

java    5920 root  mem    REG     8,3   205374  878432 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9dmp24.so

java    5920 root  mem    REG     8,3    35728  436315 /usr/lib64/libnuma.so.1

java    5920 root  mem    REG     8,3    15691  878377 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libhythr.so

java    5920 root  mem    REG     8,3    46239  878376 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libhyprtshim24.so

java    5920 root  mem    REG     8,3     6957  878403 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libvmi.so

java    5920 root  mem    REG     8,3    47241   90327 /lib64/librt-2.11.3.so

java    5920 root  mem    REG     8,3   259675  878391 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libj9prt24.so

java    5920 root  mem    REG     8,3    22489  878385 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libj9hookable24.so

java    5920 root  mem    REG     8,3    70975  878394 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libj9thr24.so

java    5920 root  mem    REG     8,3   712429  878448 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libj9vm24.so

java    5920 root  mem    REG     8,3   164469  878453 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/libjvm.so

java    5920 root  mem    REG     8,3    86110  878311 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/bin/j9vm/libjvm.so

java    5920 root  mem    REG     8,3   541821   90305 /lib64/libm-2.11.3.so

java    5920 root  mem    REG     8,3  1754140   90297 /lib64/libc-2.11.3.so

java    5920 root  mem    REG     8,3    19149   90303 /lib64/libdl-2.11.3.so

java    5920 root  mem    REG     8,3    42772  878460 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/jli/libjli.so

java    5920 root  mem    REG     8,3   135690   90323 /lib64/libpthread-2.11.3.so

java    5920 root  mem    REG     8,3   151051   90290 /lib64/ld-2.11.3.so

java    5920 root  mem    REG     8,3   217016 2906402 /var/run/nscd/passwd

java    5920 root  mem    REG     8,3    26050  576479 /usr/lib64/gconv/gconv-modules.cache

java    5920 root  mem    REG     8,3   257156  576144 /usr/lib/locale/zh_CN.utf8/LC_CTYPE

java    5920 root  mem    REG     8,3    14684  878401 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/compressedrefs/libjsig.so

java    5920 root    0u   CHR   136,0      0t0       3 /dev/pts/0

java    5920 root    1u   CHR   136,0      0t0       3 /dev/pts/0

java    5920 root    2u   CHR   136,0      0t0       3 /dev/pts/0

java    5920 root    3r   REG     8,3   416697  878426 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/amd64/default/jclSC160/vm.jar

java    5920 root    4r   REG     8,3    10160  878493 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/annotation.jar

java    5920 root    5r   REG     8,3   189574  878494 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/beans.jar

java    5920 root    6r   REG     8,3   435869  878550 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/java.util.jar

java    5920 root    7r   REG     8,3   216671  878556 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/jndi.jar

java    5920 root    8r   REG     8,3    54586  878558 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/logging.jar

java    5920 root    9r   REG     8,3   287085  878567 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/security.jar

java    5920 root   10r   REG     8,3   128157  878569 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/sql.jar

java    5920 root   11r   REG     8,3  1198418  878542 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmorb.jar

java    5920 root   12r   REG     8,3   426065  878543 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmorbapi.jar

java    5920 root   13r   REG     8,3   536255  878536 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmcfw.jar

java    5920 root   14r   REG     8,3 20511988  878566 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/rt.jar

java    5920 root   15r   REG     8,3  9601775  878496 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/charsets.jar

java    5920 root   16r   REG     8,3   501898  878565 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/resources.jar

java    5920 root   17r   REG     8,3  1161371  878544 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmpkcs.jar

java    5920 root   18r   REG     8,3   133770  878534 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmcertpathfw.jar

java    5920 root   19r   REG     8,3    27626  878538 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmjgssfw.jar

java    5920 root   20r   REG     8,3    60208  878540 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmjssefw.jar

java    5920 root   21r   REG     8,3    10966  878545 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmsaslfw.jar

java    5920 root   22r   REG     8,3    97697  878537 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmjcefw.jar

java    5920 root   23r   REG     8,3   932373  878539 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmjgssprovider.jar

java    5920 root   24r   REG     8,3   442208  878541 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmjsseprovider2.jar

java    5920 root   25r   REG     8,3   357168  878535 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmcertpathprovider.jar

java    5920 root   26r   REG     8,3   152611  878546 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ibmxmlcrypto.jar

java    5920 root   27r   REG     8,3      375  878560 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/management-agent.jar

java    5920 root   28r   REG     8,3  8915002  878575 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/xml.jar

java    5920 root   29r   REG     8,3    91126  878555 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/jlm.jar

java    5920 root   30r   REG     8,3   363834  878551 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/javascript.jar

java    5920 root   31r   REG     8,3   240130  886599 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/ibmjcefips.jar

java    5920 root   32r   REG     8,3     1273  486857 /usr/local/test/socketClient.jar

java    5920 root   33r   REG     8,3   554639  886594 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/dtfj.jar

java    5920 root   34r   REG     8,3   153450  886604 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/ibmxmlencprovider.jar

java    5920 root   35r   REG     8,3   277912  886606 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/jdmpview.jar

java    5920 root   36r   REG     8,3   481038  886589 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/IBMKeyManagementServer.jar

java    5920 root   37r   REG     8,3   232590  886601 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/ibmkeycert.jar

java    5920 root   38r   REG     8,3   183719  886588 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/CmpCrmf.jar

java    5920 root   39r   REG     8,3   206636  886598 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/ibmcmsprovider.jar

java    5920 root   40r   REG     8,3    17081  886593 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/dtfj-interface.jar

java    5920 root   41r   REG     8,3    50129  886605 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/jaccess.jar

java    5920 root   42r   REG     8,3    20228  886590 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/IBMSecureRandom.jar

java    5920 root   43r   REG     8,3    23831  886608 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/xmlencfw.jar

java    5920 root   44r   REG     8,3   353513  886602 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/ibmpkcs11impl.jar

java    5920 root   45r   REG     8,3  1202552  886596 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/gskikm.jar

java    5920 root   46r   REG     8,3   148480  886595 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/dtfjview.jar

java    5920 root   47r   REG     8,3     8230  886592 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/dnsns.jar

java    5920 root   48r   REG     8,3   112283  886597 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/healthcenter.jar

java    5920 root   49r   REG     8,3    67395  886603 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/ibmsaslprovider.jar

java    5920 root   50r   REG     8,3   992455  886600 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/ibmjceprovider.jar

java    5920 root   51r   REG     8,3    67801  886591 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/JavaDiagnosticsCollector.jar

java    5920 root   52r   REG     8,3   468772  886607 /usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre/lib/ext/localedata.jar

java    5920 root   53r   REG     8,3     1273  486857 /usr/local/test/socketClient.jar

java    5920 root   54u  IPv6 1565663      0t0     TCP node1:48707->192.168.22.15:irdmi (CLOSE_WAIT)

java    5920 root   55u  sock     0,7      0t0 1565661 can't identify protocol

java    5920 root   56u  IPv6 1571368      0t0     TCP node1:48708->192.168.22.15:irdmi (CLOSE_WAIT)

 

      node为TCP的就是两个SocketClient产生的套接字描述符的消耗，其它的99文件描述符，是跟套接字功能相关的文件打开。当Socket关闭的时候，node为TCP的套接字描述符就会别系统回收。所以，如果万一使用套接字的时候忘记关闭套接字（Scoket），不但消耗内存，更容易把系统对用户的最大文件打开数所消耗完，就会影响应用的正常运行。