java network programming

java network programming

The physical layer of the osi model is the hardware condition of network transmission: such as the optical fiber data link layer, such as the network card network layer, such as the router, which is responsible for the communication room and the search for the access address (ip protocol, routing function). The transmission layer: solves the problem of data transmission. Problem (tcp/udp) Session layer: the functional presentation layer responsible for the transmission and reception of packets and addressing of the transport layer, the communication between different types of operating systems, the translation application layer for data, and the http protocol of the tcp/ip protocol. This layer is mainly It is a specification of the data volume and data format that declares the data transmission

Application layer protocol: (2) File Transfer Protocol (File Transfer Protocol, FTP): used to implement interactive file transfer functions. (4) HyperText Transfer Protocol (HTTP): used to implement WWW services.

Transmission control protocol tcp 1, connection-oriented, reliable, byte stream-based transport layer communication protocol

2. Divide the data stream of the application layer into message segments and send them to the tcp layer of the target node

3. The data packet has a sequence number, the other party will send an ACK to confirm if it is received, and retransmit if it is not received

4. TCP has a check function to check whether there is an error in the transmission process

The header of tcp:

In a system, the difference between processes can use pid

But in the communication of the system, the difference between processes uses the source port and destination port of the tcp header to indicate the process, and the host is identified by the ip address

sequeuece number represents the message header + the number of data carried

ACK confirmation number: it is determined by sequeuece number, ACK=sequeuece number+1

offset, because the size of the data packet is uncertain, this is the offset

TCP three-way handshake process

1. The first handshake: When establishing a connection, the client sends a SYN packet [syn=j] to the server, and enters the SYN_SEND state, waiting for the server to confirm;

The second handshake: the server receives the SYN packet, it must confirm the client's SYN (ack=j+1), and at the same time it sends a SYN packet (syn=k), that is, the SYN+ACK packet, and the server enters the SYN RECV status;

The third handshake: The client receives the SYN+ACK packet from the server, and must confirm the client's SYN (ack=K+1). After the packet is sent, the client and server enter the ESTABBEL state and complete the three-way handshake.

Why do you need a three-way handshake

udp 1, non-connection-oriented,

2. It does not maintain the connection status and supports simultaneous transmission of messages to multiple clients

3. The packet header is only 8 bytes, and the extra overhead is small

4. Throughput is only limited by data production efficiency, transmission rate and machine performance

5. Do your best to deliver, do not guarantee reliable delivery, and do not need to maintain a complicated connection state table

6. Message-oriented, do not split or merge the messages submitted by the application

Conclusion: The difference between tcp and udp

Connection-oriented vs connectionless

Reliability: TCP has three handshake and four waves to provide reliability; while udp is likely to lose data

Orderliness: tcp has a sequence number, each packet can be sorted, and udp is unordered

Speed: tcp needs handshake, which consumes time, and udp is not used, so udp is suitable for multiplayer games, video and other fields that require speed

Magnitude: TCP is heavyweight, and the message has more than 20 bytes; udp is lightweight, with only 8 bytes of message

tcp sliding window

RTT and RTO

RTT: The time it takes from sending a data packet to receiving the corresponding ACK

RTO: Data packet retransmission time (data packets sent, no response, need to be retransmitted), can be calculated by RTT

TCP uses sliding windows for flow control and out-of-order reordering

1. Ensure the reliability of tcp

2. Ensure the flow control characteristics of TCP

There is a window in the header of the message, which is used by the receiver to inform the sender how much buffer area is used to receive data (to ensure that the receiver will not fail to accept it)

The calculation process of window data:

Sliding window, which is the size of the package that the cache can store

The stability of TCP is through the retransmission mechanism, that is, when no ACK from the receiver is received, it is repeatedly sent from time to time. After it is received, the window will be sliding and the subsequent packet data will be transmitted.

HTTP protocol: Hypertext transfer protocol, support client service segment mode

Features: support client/server mode, no connection, no state

HTTP request response steps

Client connects to web server

Send HTTP request

The server accepts the request and returns an HTTP response

Release TCP connection

Client browser parses HTML

HTTP status code: 1xx: the information has been accepted and is being processed

2xx: The request is returned successfully

3xx: Redirect

4xx: Client error

5xx: Service segmentation error

503 It is possible that the connection pool of the server is full, and you need to wait for a while

The difference between get and post:

Http message: the get request is placed in the url, the post request is placed in the body of the message

Added ssl to have encryption mechanism

The transport layer protocol tcp and udp tcp: 1. Connection-oriented transmission control protocol tcp 2. A connection must be established before data transmission, and the connection must be released after the data transmission is completed. Broadcast or multicast services are not provided. Because tcp must provide reliable connection-oriented Transmission services, so it inevitably adds a lot of overhead, confirmation, flow control, timer and connection management, etc. 3. Reliable, connection-oriented, long delay, suitable for large files

udp: 1. Connectionless user datagram protocol udp 2. No need to establish a connection before transmitting data, and no confirmation is required after receiving a udp message 3. Unreliable, no connection, small delay, 4. Throughput is only limited by data production efficiency, transmission rate and machine performance

The complete process of the HTTP request: 1. The browser resolves the IP address according to the domain name. Browser cache: 1. search the browser s own DNS cache (the cache time is relatively short, only about 1 minute, and can only accommodate 1000 caches), look at itself Whether there is an entry corresponding to the domain name in the cache, and it has not expired, if there is and does not expire, the resolution ends here. System cache: If the corresponding entry is not found in the browser's own cache, the browser will search the operating system's own DNS cache. If it is found and has not expired, it will stop the search and resolve to the end. Router cache: If the system cache is not found, it will send a query request to the router. ISP (Internet Service Provider) DNS cache: If it is not found in the routing cache, the last thing to check is the server where the ISP caches DNS. 2. The browser and the WEB server establish a TCP connection TCP 3-way handshake 3. The browser sends an HTTP request to the WEB server 4. The server responds to the HTTP request and returns response information 5. The browser parses the HTML code and requests HTML Resources in the code 6. Close the TCP connection, and the browser renders the page to the user

Serialization technology performance comparison: File size: hessian is the smallest, and has an advantage in transmission bandwidth. Write operation: protobuf has advantages over hessian and jdk when writing operations in large batches. Read operation: protobuf still has an advantage in reading, but in general, there is not much difference between hessian and protobuf.

Knowledge to be mastered: three-way handshake to establish a connection; 4-way handshake to disconnect; the difference between http and https; the difference between blocking and non-blocking; synchronous io and asynchronous io; this is the key difference between synchronous IO and asynchronous IO, synchronous The key difference between IO and asynchronous IO is reflected in whether the data copy phase is completed by the user thread or the kernel. So asynchronous IO must have the underlying support of the operating system.

The main steps of Socket programming based on TCP protocol Server:

  1. Construct a ServerSocket instance and specify the local port. This socket is used to monitor connection requests on the specified port.

2. Repeat the following steps: a. Call the socket accept() method to obtain the connection request of the following client. Through the socket instance returned by the accept() method, a new connection with the client is established. b. Obtain InputStream and OutputStream through this returned socket instance, and you can read and write data through these two streams respectively. c. At the end, call the close() method of the socket instance to close the socket connection. Client: 1. Construct a Socket instance and establish a connection through the specified remote server address and port. 2. Read and write data through the InputStream and OutputStream contained in the Socket instance. 3. After the operation is over, call the close method of the socket instance to close.

Open the listening port to serve: new ServerSocket(9999); accept method can create socket;

http: Stateless protocol has no memory capacity for transaction processing. The lack of status means that if the previous information is needed for subsequent processing, that is, when the client completes an HTTP request, the client sends another HTTP request, and HTTP does not know that the current client is an "old user". Cookie can be used to solve the stateless problem. Cookie is equivalent to a pass. When the client visits for the first time, a Cookie is sent to the client. When the client comes again, holding the Cookie (pass), then the server will know this Is an "old user"

29. HTTPClient related questions. What is httpClient? What is the client programming toolkit that supports HTTP protocol that HttpClient cannot do? 1. HttpClient is not a browser, it is a client-side http protocol transmission library. HttpClient is used to send and receive; 2. It will not process the content of the http message, and will not parse the content of the javascript. The Content-Type header tells the client the content type of the content actually returned.

The http protocol is an application layer protocol. The message is divided into a request message and a response message. When the client requests a web page, it will first encapsulate the requested content in the http request message through the http protocol, and the server receives the request message. After the request message, the message is parsed according to the protocol specification, and then a response message is returned to the client.

The http message structure is:

The start line describes the message. The header adds some additional information to the message. It is a name/only list. The header and the protocol work together to determine what the client and server can do together. For example: Content- Length (body length), Content-Type (body type), etc. The body contains the body part of the data

There are several models of io; the socket here is the basic socket of java communication. Blocking IO (thread block), non-blocking IO (thread always executes) (user thread access socket state), multiplexed IO (kernel access socket state, user Thread access to the kernel), signal-driven IO (the kernel will check that the request is successfully connected, and send a notification to the user thread, and the user thread connects to io) and asynchronous IO (the user thread only needs to send a request, the successful return of the request and the io connection are both the kernel Processing, the user thread can directly perform read and write operations after receiving the notification) Blocking IO (thread block), non-blocking IO (thread is always executed) Blocking IO: When the user thread sends an IO request, the kernel will check whether the data is ready If it is not ready, it will wait for the data to be ready, and the user thread will be in a blocked state, and the user thread will hand over the CPU. When the data is ready, the kernel copies the data to the user thread, and returns the result to the user thread, and the user thread releases the block state. Non-blocking IO: When the user thread initiates a read operation, it does not need to wait, but gets a result immediately. If the result is an error, it knows that the data is not ready, so it can send the read operation again. Once the data in the kernel is ready and it receives a request from the user thread again, it immediately copies the data to the user thread and then returns. So in fact, in the non-blocking IO model, the user thread needs to constantly ask whether the kernel data is ready, that is, non-blocking IO will not hand over the CPU, but will always occupy the CPU. Summary: Block the IO thread block, wait for the return value, and then start the thread; non-blocking IO constantly accesses whether the kernel data is ready. The problem of non-blocking: In the while loop, you need to constantly ask whether the kernel data is ready, which will lead to a very high CPU usage. The difference between non-blocking IO and multiplexed IO Multiplexed IO model: a model that is currently used more frequently. Java NIO is actually multiplexed IO. In the multiplexed IO model, there will be a thread that continuously polls the status of multiple sockets, and only when the socket actually has read and write events, the actual IO read and write operations are actually called. Because in the multiplexed IO model, only one thread can be used to manage multiple sockets, the system does not need to create new processes or threads, nor does it need to maintain these threads and processes, and only when there is a socket read and write event. IO resources will only be used when it is time, so it greatly reduces resource occupation. The difference between non-blocking IO (user thread access) and multiplexed IO (kernel polling): Why is multiplexed IO more efficient than the non-blocking IO model; in non-blocking IO, the user is constantly asked about the socket status through the user The thread does it, and in multiplexed IO, polling the status of each socket is done by the kernel. This efficiency is much higher than that of the user thread. 4. Signal-driven IO model (the kernel informs the thread to execute io) In the signal-driven IO model, when a user thread initiates an IO request operation, a signal function will be registered for the corresponding socket, and then the user thread will continue to execute when the kernel data is ready A signal is sent to the user thread at the time. After the user thread receives the signal, it calls the IO read and write operations in the signal function to perform the actual IO request operation. 5. Asynchronous IO model The asynchronous IO model is the most ideal IO model. In the asynchronous IO model, when a user thread initiates a read operation, it can immediately start to do other things. Therefore, no block is generated for the user thread. In the asynchronous IO model, the kernel will wait for the completion of the data preparation, and then copy the data to the user thread. When all this is completed, the kernel will send a signal to the user thread to tell it that the read operation is complete. The user thread does not need how the actual entire IO operation is performed. It only needs to initiate a request first. When the success signal returned by the kernel is received, it indicates that the IO operation has been completed and the data can be used directly. The asynchronous IO model and the signal-driven IO model are different from the asynchronous IO model. The user thread does not need to connect to the IO and can directly use the data. The successful acceptance of the request and the io connection are handled by the kernel. The signal-driven IO model also requires a connection to io Asynchronous IO model The asynchronous IO model is the most ideal IO model. In the asynchronous IO model, when a user thread initiates a read operation, it can immediately start to do other things. Therefore, no block is generated for the user thread. In the asynchronous IO model, the kernel will wait for the completion of the data preparation, and then copy the data to the user thread. When all this is completed, the kernel will send a signal to the user thread to tell it that the read operation is complete. The user thread does not need how the actual entire IO operation is performed. It only needs to initiate a request first. When the success signal returned by the kernel is received, it indicates that the IO operation has been completed and the data can be used directly. The asynchronous IO model and the signal-driven IO model are different from the asynchronous IO model. The user thread does not need to connect to the IO and can directly use the data. The successful acceptance of the request and the io connection are handled by the kernel. The signal-driven IO model also requires a connection to io

The basic development of nio realizes server-side development: 1. Open serverSocketChannel 2. Open selector 4. Set non-blocking serverSocketChannel 5. Register serverSocketChannel to selector 5. Polling the selector to monitor whether there is a connection 6. If there is a connection, traverse all Connection key 7. If the corresponding connection has been successful, 8. Bind the successfully connected socketChannel to the selector 9. Obtain the channel and buffer through the key 9. If it is readable and writable, perform read and write operations.

Client development:

  1. Get a network channel
  2. Set non-blocking mode
  3. Provide the IP address and port number of the server
  4. Connect to the server
  5. Get a buffer and store the data
  6. send data

Channel, buffer and selector understand the three main attributes of buffer: capacity: a fixed size value representing the capacity of the buffer; Buffer is full and needs to be emptied before writing; position: the meaning depends on whether the buffer is in read mode or write mode (initial The value is 0, the current position of the write or read operation) When writing data, the initial position value is 0; its maximum value can be capacity-1 Switch the Buffer from writing mode to reading mode, position will be reset to 0 limit: The meaning depends on whether the Buffer is in read mode or write mode (write limit=capacity; read limit is equal to the maximum data that can be read) In write mode, limit is equal to the capacity of Buffer. When switching Buffer to read mode, limit means you can read at most How much data

www.cnblogs.com/weijie4611/...Why do I need a three-way handshake connection? In the first handshake, A lets B know that A is connected to B. The second handshake. B lets A know that B has received the message. It proves that A can send data. The third handshake. Why do we need a four-way handshake after the data blog.csdn.net/baidu_39486...

What is the kernel: In the design of modern operating systems, in order to reduce the overhead of the system itself, some modules that are closely related to the hardware (such as interrupt handlers, device drivers, etc.), basic, common, and high-running frequency modules ( Such as clock management, process scheduling, etc.) and key data structures are separated to make them permanent in memory and protect them. Usually this part is called the kernel of the operating system.

Blocking io: When the user thread sends an IO request, the kernel will check whether the data is ready, if it is not ready, it will wait for the data to be ready, and the user thread will be in a blocked state, and the user thread will hand over the CPU. When the data is ready, the kernel copies the data to the user thread, and returns the result to the user thread, and the user thread releases the block state. An example of a typical blocking IO model is: data = socket.read(); If the data is not ready, it will always be blocked in the read method 2. Non-blocking IO model When the user thread initiates a read operation, there is no need to wait, and A result was obtained immediately. If the result is an error, it knows that the data is not ready, so it can send the read operation again. Once the data in the kernel is ready and it receives a request from the user thread again, it immediately copies the data to the user thread and then returns. So in fact, in the non-blocking IO model, the user thread needs to constantly ask whether the kernel data is ready, that is, non-blocking IO will not hand over the CPU, but will always occupy the CPU. Signal-driven IO model? Asynchronous IO model? Socket (socket) can be regarded as the endpoint reactor in each communication connection when two network applications communicate

Briefly talk about the http protocol? http is a hypertext transfer protocol, which transfers from Wanwei.com to the browser. Based on tcp/ip protocol transmission (HTML files, picture files, query results, etc.) http features: The meaning of no connection is to limit each connection to only process one request. After the server has processed the client's request and received the client's response, the HTTP protocol is a stateless protocol. Statelessness means that the protocol has no memory capacity for transaction processing.

What is the client request message under the http protocol? The request message that the client sends an HTTP request to the server includes the following format: request line (request line) request header (header) blank line request data

4.parts

The following are common HTTP status codes: l 200-the request is successful l 301-the resource (web page, etc.) is permanently transferred to another URL l 404-the requested resource (web page, etc.) does not exist l 500-internal server error

What is the nature and function of the network transmission protocol? The nature of the protocol is a transmission rule agreed upon by both parties. In order to allow the two nodes that transmit data to establish a connection, the data is transmitted and parsed according to the agreement.

Please tell me the difference between TCP and UDP? TCP application scenario: a scenario where efficiency requirements are relatively low, but accuracy requirements are relatively high. Because operations such as data confirmation, retransmission, and sorting are required during transmission, the efficiency is not as high as that of UDP. To cite a few examples: file transfer (high accuracy is required, but the speed can be relatively slow), receiving mail, remote login.

UDP application scenarios: scenarios with relatively high efficiency requirements and relatively low accuracy requirements. To cite a few examples: QQ chat, online video, Internet voice call (instant messaging, high speed requirements, but occasional intermittent is not a big problem, and the retransmission mechanism can not be used here), broadcast communication (broadcast, multiple broadcast). TCP: A connection-oriented (connection-oriented), reliable, byte stream-based Transport layer (Transport layer) communication protocol. Features: connection-oriented; point-to-point communication; high reliability; occupies a lot of system resources and low efficiency;

UDP: A connectionless transport layer communication protocol that provides transaction-oriented simple and unreliable information transmission services. Features: Non-connection-oriented transmission is unreliable and may be lost. Regardless of whether the other party is ready or not, the receiver can broadcast and send a very simple protocol with low overhead.