Tech is political: The people under attack in Palestine 🇵🇸, Iran 🇮🇷, and Lebanon 🇱🇧 are people like us. They’re our brothers and sisters, too. Read up on their history, scrutinize what you’re told, and demand that they be respected. Hide

Frontend Dogma

HTTP

on , tagged , , , , (share this post, e.g., on Mastodon or on Bluesky)

The content of this article comes from books and online resources.

I. Basic Concepts

URI

URI includes both URL and URN.

[No image text provided by the source.]

Request and Response Messages

1. Request Message

[No image text provided by the source.]

2. Response Message

[No image text provided by the source.]

II. HTTP Methods

The client sends a request message where the first line is the request line, containing the method field.

GET

Retrieve resource.

In current network requests, the vast majority use the GET method.

HEAD

Retrieve message headers.

Similar to the GET method, but does not return the message entity body part.

Mainly used to confirm URL validity and resource update date/time, etc.

POST

Transmit entity body.

POST is mainly used to transmit data, while GET is mainly used to retrieve resources.

For more comparisons between POST and GET, please refer to Chapter IX.

PUT

Upload file.

Since it does not have its own authentication mechanism, anyone can upload files, which creates security issues; therefore, this method is generally not used.

PUT /new.html HTTP/1.1
Host: example.com
Content-type: text/html
Content-length: 16

<p>New File</p>

PATCH

Perform partial modifications to resources.

PUT can also be used to modify resources, but can only completely replace the original resource; PATCH allows partial modifications.

PATCH /file.txt HTTP/1.1
Host: www.example.com
Content-Type: application/example
If-Match: "e0023aa4e"
Content-Length: 100

[description of changes]

DELETE

Delete file.

Opposite function to PUT, and similarly lacks an authentication mechanism.

DELETE /file.html HTTP/1.1

OPTIONS

Query supported methods.

Query what methods the specified URL supports.

Will return content like Allow: GET, POST, HEAD, OPTIONS.

CONNECT

Require establishing a tunnel when communicating with proxy servers.

Uses SSL (Secure Sockets Layer) and TLS (Transport Layer Security) protocols to encrypt communication content and transmit it over network tunnels.

CONNECT www.example.com:443 HTTP/1.1

[No image text provided by the source.]

TRACE

Trace path.

The server returns the communication path back to the client.

When sending a request, enter a value in the Max-Forwards header field; it decreases by 1 each time it passes through a server, and transmission stops when the value reaches 0.

TRACE is typically not used, and it is susceptible to XST attacks (Cross-Site Tracing).

III. HTTP Status Codes

The response message returned by the server has a status line as its first line, containing the status code and reason phrase, which informs the client of the request result.

Status CodeCategoryMeaning
1XXInformationalReceived request is being processed
2XXSuccessRequest processed successfully
3XXRedirectionAdditional action needed to complete request
4XXClient ErrorServer cannot process request
5XXServer ErrorServer encountered error processing request

1XX Informational

2XX Success

3XX Redirection

4XX Client Error

5XX Server Error

IV. HTTP Headers

There are four types of header fields: General Header Fields, Request Header Fields, Response Header Fields, and Entity Header Fields.

Various header fields and their meanings are as follows (full memorization not required; for reference only):

General Header Fields

Header Field NameDescription
Cache-ControlControls caching behavior
ConnectionControls header fields not forwarded to proxies; manages persistent connections
DateDate and time when the message was created
PragmaMessage directives
TrailerOverview of headers at the end of the message
Transfer-EncodingSpecifies the transfer encoding method for the message body
UpgradeUpgrades to another protocol
ViaInformation about proxy servers
WarningError notification

Request Header Fields

Header Field NameDescription
AcceptMedia types that the user agent can handle
Accept-CharsetPreferred character sets
Accept-EncodingPreferred content encodings
Accept-LanguagePreferred languages (natural language)
AuthorizationWeb authentication information
ExpectExpects specific behavior from the server
FromUser’s email address
HostServer where the requested resource resides
If-MatchCompares entity tags (ETag)
If-Modified-SinceCompares resource update time
If-None-MatchCompares entity tags (opposite of If-Match)
If-RangeSends entity byte range request if resource not updated
If-Unmodified-SinceCompares resource update time (opposite of If-Modified-Since)
Max-ForwardsMaximum transmission hop count
Proxy-AuthorizationAuthentication information for proxy server from client
RangeEntity byte range request
RefererOriginal source of the URI in the request
TETransfer encoding priority
User-AgentHTTP client program information

Response Header Fields

Header Field NameDescription
Accept-RangesWhether byte range requests are accepted
AgeEstimated time elapsed since resource creation
ETagResource matching information
LocationRedirects client to specified URI
Proxy-AuthenticateProxy server authentication information for client
Retry-AfterTiming requirements for re-requesting
ServerHTTP server installation information
VaryProxy server cache management information
WWW-AuthenticateServer authentication information for client

Entity Header Fields

Header Field NameDescription
AllowHTTP methods supported by the resource
Content-EncodingEncoding method applicable to the entity body
Content-LanguageNatural language of the entity body
Content-LengthSize of the entity body
Content-LocationAlternative URI for the corresponding resource
Content-MD5Message digest of the entity body
Content-RangePosition range of the entity body
Content-TypeMedia type of the entity body
ExpiresExpiration date and time of the entity body
Last-ModifiedLast modification date and time of the resource

V. Practical Applications

Connection Management

[No image text provided by the source.]

1. Short Connections and Long Connections

When a browser accesses an HTML page containing multiple images, in addition to requesting the HTML page resource, it also requests image resources. If a new TCP connection is created for each HTTP communication, the overhead is substantial.

Long connections only need one TCP connection established to conduct multiple HTTP communications.

2. Pipelining

By default, HTTP requests are sent sequentially; the next request is only sent after the current request receives a response. Due to network latency and bandwidth limitations, a long wait may be required before the next request is sent to the server.

Pipelining sends requests continuously on the same long connection without waiting for responses to return, thereby reducing latency.

Cookie

The HTTP protocol is stateless, primarily to keep the protocol as simple as possible, enabling it to handle massive transactions. HTTP/1.1 introduced cookies to preserve state information.

A cookie is a small piece of data sent by the server to the user’s browser and saved locally; it will be carried when the browser subsequently initiates a request to the same server, informing the server whether two requests originate from the same browser. Since each subsequent request needs to carry cookie data, this creates additional performance overhead (especially in mobile environments).

Cookies were once used for client-side data storage because there were no suitable storage alternatives at the time, serving as the sole storage mechanism. However, with modern browsers now supporting various storage methods, cookies are gradually being phased out. New browser APIs now allow developers to store data directly locally, such as using the Web Storage API (local storage and session storage) or IndexedDB.

1. Usage

2. Creation Process

The response message sent by the server contains the Set-Cookie header field; after receiving the response message, the client saves the cookie content in the browser.

HTTP/1.0 200 OK
Content-type: text/html
Set-Cookie: yummy_cookie=choco
Set-Cookie: tasty_cookie=strawberry

[page content]

When the client subsequently sends a request to the same server, it retrieves cookie information from the browser and sends it to the server via the Cookie request header field.

GET /sample_page.html HTTP/1.1
Host: www.example.org
Cookie: yummy_cookie=choco; tasty_cookie=strawberry

3. Classification

Set-Cookie: id=a3fWa; Expires=Wed, 21 Oct 2015 07:28:00 GMT;

4. Scope

Domain specifies which hosts can accept cookies. If not specified, the default is the current document’s host (excluding subdomains). If Domain is specified, it generally includes subdomains. For example, if setting Domain=mozilla.org, the cookie also applies to subdomains (such as developer.mozilla.org).

Path specifies which paths under the host can accept cookies (the URL path must exist in the request URL). The character %x2F (“/”) serves as the path separator; subpaths will also be matched. For example, setting Path=/docs means the following addresses will match:

5. JavaScript

Browsers can create new cookies through the document.cookie attribute, and can also access non-HttpOnly-marked cookies through this attribute.

document.cookie = "yummy_cookie=choco";
document.cookie = "tasty_cookie=strawberry";
console.log(document.cookie);

6. HttpOnly

Cookies marked as HttpOnly cannot be called by JavaScript scripts. Cross-site scripting attacks (XSS) often use JavaScript’s document.cookie API to steal users’ cookie information; therefore, using the HttpOnly mark can avoid XSS attacks to some extent.

Set-Cookie: id=a3fWa; Expires=Wed, 21 Oct 2015 07:28:00 GMT; Secure; HttpOnly

7. Secure

Cookies marked as Secure can only be sent to the server through requests encrypted by the HTTPS protocol. However, even with the Secure mark set, sensitive information should not be transmitted through cookies, because cookies have inherent insecurity, and the Secure mark cannot provide definitive security guarantees.

8. Session

Besides storing user information in the user’s browser via cookies, session can also store information on the server side, which is more secure.

Session can be stored in files, databases, or memory on the server. Session can also be stored in in-memory databases such as Redis, which offers higher efficiency.

The process of using session to maintain user login status is as follows:

Security issues regarding Session ID should be noted—it must not be easily obtained by malicious attackers, meaning Session ID values should not be easily guessable. Additionally, Session IDs need frequent regeneration. In scenarios requiring extremely high security, such as fund transfers, besides using session to manage user status, users also need re-verification, such as re-entering passwords or using SMS verification codes, etc.

9. Browser Disabled Cookies

At this point, cookies cannot be used to save user information; only sessions can be used. Furthermore, Session IDs can no longer be stored in cookies; instead, URL rewriting technology is used to pass Session IDs as URL parameters.

10. Cookie Versus Session Selection

Caching

1. Advantages

2. Implementation Methods

3. Cache-Control

HTTP/1.1 controls caching through the Cache-Control header field.

3.1 Prohibiting Caching

The no-store directive stipulates that no part of requests or responses can be cached.

Cache-Control: no-store
3.2 Forced Cache Confirmation

The no-cache directive stipulates that cache servers must first validate cache resource effectiveness with the origin server; only when the cache resource is valid can the cache respond to client requests.

Cache-Control: no-cache
3.3 Private Caches and Public Caches

The private directive stipulates resources as private caches, usable only by individual users, generally stored in user browsers.

Cache-Control: private

The public directive stipulates resources as public caches, usable by multiple users, generally stored in proxy servers.

Cache-Control: public
3.4 Cache Expiration Mechanism

The max-age directive appearing in request messages accepts the cache if the cache resource’s caching time is less than the time specified by this directive.

The max-age directive appearing in response messages indicates how long the cache resource is retained in the cache server.

Cache-Control: max-age=31536000

The Expires header field can also be used to inform cache servers when the resource expires.

Expires: Wed, 04 Jul 2012 08:26:05 GMT

4. Cache Validation

The ETag header field needs to be understood first; it is the unique identifier for resources. URLs cannot uniquely represent resources—for example, “http://www.google.com/” has Chinese and English versions as resources; only ETag can uniquely identify these two resources.

ETag: "82e22293907ce725faf67773957acd12"

The cached resource’s ETag value can be placed in the If-None-Match header; after receiving this request, the server compares whether the cached resource’s ETag value matches the latest resource ETag value. If consistent, the cached resource is valid, and the server returns 304 Not Modified.

If-None-Match: "82e22293907ce725faf67773957acd12"

The Last-Modified header field can also be used for cache validation; it is contained in response messages sent by the origin server, indicating the origin server’s last modification time of the resource. However, it is a weak validator because it only achieves one-second precision, so it typically serves as a backup to ETag. If this information exists in the response header field, clients can include If-Modified-Since in subsequent requests to validate caches. The server only returns the resource if it was modified after the given date/time, with status code 200 OK. If the requested resource has not been modified since that time, a 304 Not Modified response message without an entity body is returned.

Last-Modified: Wed, 21 Oct 2015 07:28:00 GMT
If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMT

Content Negotiation

Returns the most appropriate content through content negotiation, such as choosing whether to return a Chinese or English interface based on the browser’s default language.

1. Types

1.1 Server-Driven Type

The client sets specific HTTP header fields, such as Accept, Accept-Charset, Accept-Encoding, and Accept-Language; the server returns specific resources based on these fields.

It has the following problems:

1.2 Proxy-Driven Type

The server returns 300 Multiple Choices or 406 Not Acceptable; the client selects the most appropriate resource from them.

2. Vary

Vary: Accept-Language

When using content negotiation, the cache can only be used when the content in the cache server meets content negotiation conditions; otherwise, the origin server should be requested for the resource.

For example, after a client sends a request containing the Accept-Language header field, the origin server’s response contains Vary: Accept-Language content. After the cache server caches this response, when the client next accesses the same URL resource and the Accept-Language matches the corresponding value in the cache, the cache is returned.

Content Encoding

Content encoding compresses the entity body, thereby reducing transmitted data volume.

Common content encodings include: gzip, compress, deflate, and identity.

Browsers send Accept-Encoding headers, containing supported compression algorithms along with respective priorities. The server selects one algorithm from them, uses it to compress the response message body, and sends the Content-Encoding header to inform the browser which algorithm was chosen. Since this content negotiation process selects resource display forms based on encoding types, the response message’s Vary header field should at least include Content-Encoding.

Range Requests

If network interruption occurs and the server only sends partial data, range requests enable the client to request only the unsent portion from the server, avoiding the need for the server to resend all data.

1. Range

Add the Range header field in the request message to specify the request range.

GET /z4d4kWk.jpg HTTP/1.1
Host: i.imgur.com
Range: bytes=0-1023

If the request succeeds, the server returns a response containing status code 206 Partial Content.

HTTP/1.1 206 Partial Content
Content-Range: bytes 0-1023/146515
Content-Length: 1024
…
(binary content)

2. Accept-Ranges

The response header field Accept-Ranges informs the client whether it can handle range requests; use bytes if it can, or none otherwise.

Accept-Ranges: bytes

3. Response Status Codes

Chunked Transfer Coding

Chunked Transfer Encoding can split data into multiple blocks, allowing browsers to display pages progressively.

Multipart Object Collections

Multiple types of entities can be contained within one message body and sent simultaneously; each part is separated using delimiters defined by the boundary field, and each part can have header fields.

For example, uploading multiple forms can use the following method:

Content-Type: multipart/form-data; boundary=AaB03x

--AaB03x
Content-Disposition: form-data; name="submit-name"

Larry
--AaB03x
Content-Disposition: form-data; name="files"; filename="file1.txt"
Content-Type: text/plain

… contents of file1.txt …
--AaB03x--

Virtual Hosts

HTTP/1.1 uses virtual host technology, enabling one server to possess multiple domains, logically appearing as multiple servers.

Communication Data Forwarding

1. Proxy

Proxy servers accept client requests and forward them to other servers.

The main purposes of using proxies are:

Proxy servers are divided into forward proxies and reverse proxies:

2. Gateway

Unlike proxy servers, gateway servers convert HTTP to other protocols for communication, thus requesting services from other non-HTTP servers.

3. Tunnel

Uses encryption methods such as SSL to establish a secure communication channel between the client and server.

VI. HTTPS

HTTP has the following security issues:

HTTPS is not a new protocol; rather, it lets HTTP communicate with SSL (Secure Sockets Layer) first, then SSL communicates with TCP. In other words, HTTPS uses tunnels for communication.

Through SSL, HTTPS possesses encryption (prevents eavesdropping), authentication (prevents impersonation), and integrity protection (prevents tampering).

[No image text provided by the source.]

Encryption

1. Symmetric Key Encryption

Symmetric-key encryption uses the same key for both encryption and decryption.

[No image text provided by the source.]

2. Asymmetric Key Encryption

Asymmetric key encryption, also known as public-key encryption, uses different keys for encryption and decryption.

Anyone can obtain the public key; after the communication sender obtains the receiver’s public key, encryption can be performed using the public key; the receiver decrypts the communication content after receipt using the private key.

Asymmetric keys can also be used for signing besides encryption. Since private keys cannot be obtained by others, the communication sender signs using their private key; the communication receiver decrypts the signature using the sender’s public key to determine whether the signature is correct.

[No image text provided by the source.]

3. Encryption Method Used By HTTPS

As mentioned above, symmetric key encryption has higher transmission efficiency but cannot securely transmit the secret key to the communication party. Asymmetric key encryption ensures transmission security, so we can use asymmetric key encryption to transmit secret keys to the communication party. HTTPS adopts a hybrid encryption mechanism, utilizing the scheme mentioned above:

[No image text provided by the source.]

Authentication

Uses certificates to authenticate communication parties.

Digital Certificate Authorities (CA, Certificate Authority) are trusted third-party institutions for both clients and servers.

Server operators submit applications for public keys to the CA; after confirming the applicant’s identity, the CA digitally signs the applied public key, then distributes this signed public key and binds it into a public key certificate.

During HTTPS communication, the server sends the certificate to the client. After obtaining the public key from the certificate, the client first verifies using digital signatures; if verification passes, communication can begin.

[No image text provided by the source.]

Integrity Protection

SSL provides message digest functionality for integrity protection.

HTTP also provides MD5 message digest functionality, but it is not secure. For example, after message content is tampered with, recalculating the MD5 value means the communication receiver cannot realize that tampering occurred.

HTTPS’s message digest functionality is secure because it combines encryption and authentication operations. Imagine encrypted messages—after being tampered with, recalculating the message digest is difficult because plaintext cannot be easily obtained.

Disadvantages of HTTPS

VII. HTTP/2.0

HTTP/1.x Defects

HTTP/1.x simplicity comes at the cost of performance:

Binary Framing Layer

HTTP/2.0 splits messages into HEADERS frames and DATA frames; both are binary formats.

[No image text provided by the source.]

During communication, only one TCP connection exists, carrying any number of bidirectional data streams.

[No image text provided by the source.]

Server Push

When HTTP/2.0 clients request a resource, related resources are sent to the client simultaneously, eliminating the need for subsequent requests. For example, when clients request the page.html page, the server sends related resources like script.js and style.css to the client simultaneously.

[No image text provided by the source.]

Header Compression

HTTP/1.1 headers carry a lot of information and must be resent repeatedly.

HTTP/2.0 requires clients and servers to simultaneously maintain and update a table containing previously seen header fields, avoiding duplicate transmissions.

Furthermore, HTTP/2.0 also uses Huffman coding to compress header fields.

[No image text provided by the source.]

VIII. HTTP/1.1 New Features

For detailed content, please refer to the text above.

IX. GET and POST Comparison

Purpose

GET is used to retrieve resources, while POST is used to transmit entity bodies.

Parameters

Both GET and POST requests can use additional parameters, but GET parameters appear as query strings in the URL, while POST parameters are stored in the entity body. One should not believe POST parameters are more secure simply because they are stored in the entity body, since packet capture tools (like Fiddler) can still view them.

Because URLs only support ASCII characters, GET parameters containing characters such as Chinese must be encoded first. For example, 中文 converts to %E4%B8%AD%E6%96%87, and spaces convert to %20. POST parameters support standard character sets.

GET /test/demo_form.asp?name1=value1&name2=value2 HTTP/1.1
POST /test/demo_form.asp HTTP/1.1
Host: w3schools.com
name1=value1&name2=value2

Security

Secure HTTP methods do not change server state; they are read-only.

The GET method is secure, while POST is not, because POST’s purpose is to transmit entity body content; this content may be form data uploaded by users; after successful upload, the server may store this data in a database, thus changing state.

Secure methods besides GET also include: HEAD, OPTIONS.

Insecure methods besides POST also include: PUT, DELETE.

Idempotency

Idempotent HTTP methods produce the same effect whether executed once or consecutively multiple times; server state is identical. In other words, idempotent methods should not have side effects (except for statistical purposes).

All safe methods are also idempotent.

Under correct implementation conditions, GET, HEAD, PUT, and DELETE methods are idempotent, while the POST method is not.

GET /pageX HTTP/1.1 is idempotent; calling it consecutively multiple times produces the same result received by the client:

GET /pageX HTTP/1.1
GET /pageX HTTP/1.1
GET /pageX HTTP/1.1
GET /pageX HTTP/1.1

POST /add_row HTTP/1.1 is not idempotent; calling it multiple times adds multiple record rows:

POST /add_row HTTP/1.1   → Adds a 1st row
POST /add_row HTTP/1.1   → Adds a 2nd row
POST /add_row HTTP/1.1   → Adds a 3rd row

DELETE /idX/delete HTTP/1.1 is idempotent, even if different requests receive different status codes:

DELETE /idX/delete HTTP/1.1   → Returns 200 if “idX” exists
DELETE /idX/delete HTTP/1.1   → Returns 404 as it just got deleted
DELETE /idX/delete HTTP/1.1   → Returns 404

Cacheable

To cache responses, the following conditions must be met:

XMLHttpRequest

To illustrate another distinction between POST and GET, XMLHttpRequest must first be understood:

XMLHttpRequest is an API that provides functionality for transmitting data between client and server. It offers a simple way to retrieve data through URLs without refreshing the entire page. This allows web pages to update only part of the page without disturbing the user. XMLHttpRequest is extensively used in AJAX.

References

(This post is a machine-made, human-reviewed, and authorized translation of xuxueli.com/­blog/­?blog=network/­http.)