Introduction to the AMQP protocol
AMQP (Advanced Message Queuing Protocol) is an application-layer standard protocol that provides unified messaging services, and is an open standard for application-layer protocols designed for message-oriented middleware. AMQP is a network protocol for passing asynchronous messages between processes.
Clients and message middleware based on this protocol can deliver messages without being restricted by different client/middleware products, different development languages, etc.
The main characteristics of AMQP are message-oriented, queued, routing (including peer-to-peer and publish/subscribe), reliability, and security. AMQP enforces the behavior of message providers and clients, enabling true interoperability between different vendors.
AMQP and JMS
JMS was an attempt to standardize early message middleware, it was only standardized at the API level, and it was far from creating interoperability.
Unlike JMS, AMQP is a wire-level protocol that describes the format of data transmitted over a network, flowing in bytes. As a result, any tool that adheres to this data format, which creates and interprets messages, is interoperable with other compatible tools.
AMQP core composition
Producer
Production news.
ConnectionFactory
The factory that manufactures Connection.
Connection
Connection, application network connection with Broker TCP/IP/Triple Handshake and Quad Wave.
AMQP connections are usually long connections. AMQP is an application-layer protocol that uses TCP to provide reliable delivery. AMQP uses authentication mechanisms and provides TLS (SSL) protection. When an application no longer needs to connect to the AMQP proxy, it needs to gracefully release the AMQP connection instead of simply shutting down the TCP connection.
Channel
Network channel is a lightweight connection built on top of the Connection. Almost all operations are performed in Channels, which are channels for reading and writing messages, and clients can establish pairs for each channel, each of which represents a session task.
If the Connection is compared to a fiber optic cable, then the Channel channel is compared to one of the fibers in a fiber optic cable. Any number of channels can be created on a Connection.
Most of our business operations are done in the Channel interface, including:
- queueDeclare
- The exchangeDeclare for the switch
- queueBind queueBind
- Publish the message basicPublish
- Consumer newsbasicConsume, etc.
Broker
Accept the client's connection to implement AMQP entity services, such as rabbitmq.
VirtualHost (Web Hosting)
Virtual hosting, used for logical isolation, a virtual host can have several exchanges and queues, and the same virtual host cannot have exchanges with the same name.
To implement multiple isolated environments (users, user groups, switches, queues, etc.) on a single proxy, AMQP provides the concept of virtual hosts (virtual hosts - vhosts). This is very similar to the Web servers web hosting concept, which provides a completely isolated environment for AMQP entities. When the connection is established, the AMQP client specifies which virtual host to use.
Exchange
The switch, accepts messages, and sends messages to the bound queue based on the routing key (without message storage capabilities).
A switch is an AMQP entity used to send messages. After the switch gets a message, it routes it to one or zero queues. The routing algorithm it uses is determined by the switch type and the binding rules.
Switch Type:
- Direct exchange
- Fanout exchange
- Topic exchange
- Headers exchange
Switch Properties:
- Name: The name of the switch
- Durability: A persistence flag that indicates whether this switch is persisted or not
- Auto-delete: Delete flag, indicatingWhen all queues are done using this exchange, whether they are deleted
- Arguments: Dependent on the agent itself
Switch Status:
Persistent switches will exist after the broker is restarted, while staging switches will not (they will need to be re-declared after the broker is back online).
Default switch
The default exchange is actually a direct exchange that is pre-declared by the message broker and has no name (the name is an empty string).
You can think of the default switch as a special direct-attached switch.
Default switch name: Null string (AMQP default)
Default switch type: Direct-attached switch
When creating a queue, as long as the switch to be bound to be bound is not specified, it will be automatically bound to the default switch, and the routing key name of the binding will be the same as the queue name.
Direct connection to the switch
A direct-attached switch delivers messages to a queue of corresponding binding keys based on the routing key carried by the message. The unicast routing used by the direct switch to handle the message.
When creating a Queue, if it is bound to a direct switch, it does not need to specify the name of the routing key, because it will have a default routing key name, which is the same as the Queue name.
A queue of direct-attached switches typically distributes tasks to multiple consumers in a loop (we call this polling).
Workflow:
- When binding a queue to a switch, give it a binding key, assuming R;
- When a message with a Routing Key is sent to a direct-attached switch, the switch routes it to a queue with a Routing Key.
Fan switches
The fan switch routes messages to all queues bound to it, regardless of the bound routing key.
If N queues are bound to a sector switch, when a message is sent to this sector switch, the switch sends a copy of the message to all N queues separately. Fan switches are generally used to handle broadcast routing of messages.
Application Scenarios:
broadcast messages;
Group chat function.
Theme switch
The topic switch sends messages to one or more Queues according to the routing key and the type of Exchange, and we often use it to implement various publish/subscribe, that is, publish subscriptions.
The routing rules for direct-attached switches are strictly matched, which means that the Routing Key must match the Binding Key before sending a message to the Queue.
The routing rules of the topic switch are fuzzy matches that can be delivered by satisfying some of the rules through wildcards.
It stipulates:
- There can be two special characters * and # in the binding key for fuzzy matching. where * is used to match a word, #用于匹配多个单词 (can be zero)
- A routing key is a dot-separated string (we call each individual string separated by a dot mark a word)
- When the producer sends the message Routing Key=A.A.A, only A.*.* is satisfied, and it will only be routed to queue1.
- When the producer sends the message Routing Key=A.B.A, satisfying A.*.* and *.B.* will be routed to queue1 and queue2.
- When the producer sends the message Routing Key=A.B.C, then A.*.* and *.B.* and *.* are satisfied. C is routed to queue1, queue2, queue3.
Application Scenarios:
- news updates involving categories or tags;
- Background tasks completed by multiple workers, each of whom is responsible for handling certain specific tasks.
Head switch
Headers switches do not rely on the matching rules of the routing key to binding keys to route messages, but rather match based on the headers attribute in the content of the sent message.
Head switches can be considered as another manifestation of a direct-attached switch. However, the routing key of a direct switch must be a string, and the header attribute values are not constrained by this, they can even be integers or hash values (dictionaries), etc. More flexibility (but in practice we rarely use head switches).
Workflow:
- When a queue is bound to a header switch, multiple headers are bound at the same time for matching.
- Incoming messages carry a header and an "x-match" parameter. When "x-match" is set to "any", any value of the header can be matched, and when "x-match" is set to "all", all values of the header must be matched.
Switch Summary
Binding
Virtual connection between Exchange and Queue.
BindingKey is a rule description for Exchange and Queue bindings. The binding key specifies what kind of Routing Key will be assigned to the currently bound Queue under the current exchange.
Routing Key
Routing rules, which the virtual machine can use to determine how to route a particular message.
Binding Key vs. Routing Key
- The binding key is the binding key between the queue and the switch;
- The Routing Key is a piece of information sent by the producer to the switch;
- When the Binding Key and Routing Key correspond, put the message in the corresponding queue.
Binding Key is the rule description of Exchange and Queue binding, which is used to parse when Exchange receives a message, the message received by Exchange will have a Routing Key field, and Exchange matches this Routing Key with all binding Keys of the current Exchange, and if the requirements are met, it will be sent to Binding Key is bound to the Queue to send the message.
Binding Key vs. Routing Key in various switches
Default switch: Binding Key is the Queue name, which cannot be customized. The Routing Key is also the Queue name before it can be successfully routed to the queue
Direct connection switch: Binding Key is the Queue name, which can be customized. Routing Keys can only be successfully routed to the queue when the Binding Key is the same
Fan switch: No binding key; Of course, there is no Routing Key. Automatically routed to all queues bound to the switch
Theme switch: custom Binding Key; Customize the Routing Key. Routing Key==Binding Key, and the fuzzy match must be successfully routed to the queue
Head switch: no binding key; Of course, there is no Routing Key. Matches based on the headers attribute in the content of the message sent
Queue
Stores messages that are about to be consumed by the app.
Queue Properties:
- Name: The name of the queue
- Durable: The queue still exists after the message broker is restarted
- Exclusive: Used by only one connection, and the queue is deleted when the connection is closed
- Auto-delete: Deleted when the last consumer unsubscribes
- Arguments: Some message brokers use it to do some extra functions similar to TTL
Queue creation:
Queues can only be used after they are declared. If a queue doesn't already exist, declaring a queue creates it. If the declared queue already exists and the attributes are identical, the declaration has no effect on the original queue. If the attributes in the declaration differ from those in the existing queue, a channel-level exception with an error code of 406 is thrown.
Queue persistence:
The persistence queue is stored on disk and remains there when the broker is restarted. Queues that are not persisted are called transient queues. Not all scenarios and cases require queue persistence.
A persisted queue does not make messages routed to it persistent. If the message agent hangs up and is restarted, the persistence queue will be re-declared during the restart, and in any case, only persisted messages can be re-restored.
Consumer
Consumer Consumption News. In AMQP, there are two ways for consumers to get pending messages:
Message middleware delivers messages to consumers (push API)
Consumers actively get messages (pull API)
Note: When multiple consumers listen to the same queue, the messages in the queue will only be consumed by one of the consumers (not once for each consumer)
Message
The data transmitted between messages, services, and applications consists of Properties and bodies.
Attributes modify messages, such as message priority, delay and other advanced features, and the main body is the content of the message body.
Message Properties:
- Content type
- Content encoding
- Routing key
- Delivery mode (persistent or not)
- Delivery mode (persistent or non-persistent)
- Message priority
- Message publishing timestamp
- Expiration period
- Publisher application id
Message body:
In addition to the attributes, AMQP messages also contain a payload (the data that the message actually carries), which is treated by the AMQP proxy as an opaque array of bytes.
The message broker does not inspect or modify the payload. Messages can contain only attributes without carrying a payload. It typically uses data in a serialized format like JSON, and to save money, protocol buffers and MessagePacks will serialize the structured data for publishing as a payload of messages. AMQP and its peers typically use the "content-type" and "content-encoding" fields to communicate with messages to identify payloads, but this is only based on conventions.
Message persistence:
Messages are published in a persistent manner, and the AMQP agent stores this message on disk. If the server is restarted, the system confirms that the persistence message received is not lost.
Simply sending a message to a persistent switch or routing it to a persisted queue does not make the message persistent: the persistence of the message depends entirely on the persistence mode of the message itself.
Publishing messages in a persistent manner can have a performance impact.
AMQP work process
The Publisher publishes a message through the Exchange.
The switch distributes the incoming messages to the queue bound to the switch according to the routing rules.
Finally, the AMQP agent will deliver the message to the consumer who has subscribed to this queue, or the consumer will get it on their own as needed.
AMQP messaging mechanism
Message confirmation
Consumers occasionally fail to process messages or sometimes crash directly. And network reasons can also cause various problems.
This presents us with the question of when is the correct time for AMQP agents to delete messages.
AMQP's two message acknowledgment modes:
Auto-Confirm Mode: Delete the message as soon as it is sent to the consumer by the message middleware. (Using the AMQP method: basic.deliver or basic.get-ok)
Explicit confirmation mode: Wait for the consumer to send an acknowledgement before deleting the message. (Using AMQP method: basic.ack)
If a consumer hangs up without sending a confirmation receipt, the AMQP agent redelivers the message to another consumer. If there are no consumers available at that time, the message broker waits for the next consumer to register in this queue and then tries to deliver again.
Reject messages
When a consumer receives a message, the processing process may succeed or fail. The consumer can indicate to the message broker (message middleware) that the message failed to be processed (or failed to complete at this point) due to a "rejected message".
When a message is rejected, the consumer can tell the message broker what to do with the message - destroy it or put it back in the queue.
When there is only one consumer in this queue, make sure that you do not reject the message and choose to put it back in the queue, causing the message to loop indefinitely on the same consumer.
In AMQP, the basic.reject method is used to perform the operation of rejecting messages. However, basic.reject has a limitation: you can't use it to reject multiple messages with acknowledgements. But if you're using RabbitMQ, you can use the AMQP 0-9-1 extension called negative acknowledgements (also called nacks) to solve this problem.
Original:The hyperlink login is visible.
|
Posted on 8/14/2022 11:22:55 PM
|
|
|
|
