How to Use RabbitMQ Message Queues in Magento 2?

RabbitMQ is a message broker that helps Magento 2 handle asynchronous messaging between different modules or services. It can be used for background tasks in Magento 2, reducing the load on your web server, which improves performance and scalability.

RabbitMQ plays a crucial role in the Magento 2 architecture by providing a scalable and dependable messaging system. This allows the platform to manage complex tasks and handle high traffic volumes efficiently.

Optimizing Magento 2 Performance with RabbitMQ Integration

Here’s how RabbitMQ is leveraged in Magento 2:

1. Asynchronous Processing

RabbitMQ allows tasks to be processed in the background without making users wait for them to complete. For example, Magento 2 can offload time-consuming processes like sending order confirmation emails or processing inventory updates into RabbitMQ queues, improving the platform’s responsiveness.

2. Decoupling Components

With RabbitMQ, different parts of Magento 2 (such as order management, shipping, and inventory) can communicate together without being tightly integrated. This improves flexibility, as changes in one part of the system won’t directly impact others.

3. Handling High Traffic and Heavy Loads

RabbitMQ helps Magento 2 scale by distributing tasks across multiple consumers (workers), allowing the system to handle higher loads more efficiently. This is critical for ecommerce platforms facing large traffic or spikes during sales events.

4. Queue Management

RabbitMQ uses queues to store messages (tasks) until they can be processed. Magento 2 can create different queues for different tasks (e.g., order processing, and shipping updates), which enhances system organization and efficiency.

5. Failure Handling

RabbitMQ ensures that tasks in the queue are not lost in case of any system failures. Messages will be re-queued and processed when the system is back up, ensuring reliability.

6. Improved Performance

By offloading tasks to RabbitMQ, Magento 2 improves front-end performance and user experience since heavy backend tasks don’t block user interactions.

8 Steps to Set Up RabbitMQ in Magento 2

Here’s an example of how you can configure RabbitMQ in Magento 2 and set up a custom message queue.

• RabbitMQ Installation: Install RabbitMQ on your server or local machine. 
• Magento 2 Configuration: Configure Magento 2 to use RabbitMQ by modifying the env.php file.
• Add the RabbitMQ configuration in app/etc/env.php

'queue' => [
    'amqp' => [
        'host' => 'rabbitmq_host',
        'port' => '5672',
        'user' => 'guest',      // default user
        'password' => 'guest',  // default password
        'virtualhost' => '/'
    ]
]

Replace rabbitmq_host with the actual host where RabbitMQ is running. If RabbitMQ is installed locally, you can use localhost.

Also read- Custom Filter Attribute in Category Graphql in Magento 2

Let’s create a module to set up a custom message queue.

Step 1: Create the Module Registration File

You need to register your custom module using registration.php.

Path: app/code/Klizer/QueueExample/registration.php

<?php
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'Klizer_QueueExample',
    __DIR__
);

Step 2: Define the Module in module.xml

Path: app/code/Klizer/QueueExample/etc/module.xml

<?xml version="1.0"?>
<!--
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
*/
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="Klizer_QueueExample" setup_version="1.0.0">
    </module>
</config>

Step 3: Create communication.xml

Use: Defines aspects of the message queue system that all communication types have in common.

Path: app/code/Klizer/QueueExample/etc/communication.xml

<?xml version="1.0"?>
<!--
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
*/
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
    <topic name="klizer.queue.example" request="string"/>
</config>

Step 4: Create queue_topology.xml

Use: Defines the message routing rules and declares queues and exchanges.

Path: app/code/Klizer/QueueExample/etc/queue_topology.xml

<?xml version="1.0"?>
<!--
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
*/
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/queue_topology.xsd">
    <!-- name : A unique ID for the exchange  -->
    <!-- type : Specifies the type of exchange. Must be topic -->
    <!-- connection: For AMQP connections, a string that identifies the connection. For MySQL connections, the connection name must be db -->
    <exchange name="klizer.queue.example" type="topic" connection="amqp">
        <!-- id: A unique ID for this binding -->
        <!-- topic: The name of a topic -->
        <!-- destinationType: Must be queue -->
        <!-- destination: Identifies the name of a queue -->
        <binding id="klizer_queue_example" topic="klizer.queue.example" destinationType="queue" destination="klizer_queue_example"/>
    </exchange>
</config>

Step 5: Create queue_consumer.xml

Use: Defines the relationship between an existing queue and its consumer.

Path: app/code/Klizer/QueueExample/etc/queue_consumer.xml

<?xml version="1.0"?>
<!--
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
*/
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/queue_consumer.xsd">
    <!-- name: The name of the consumer -->
    <!-- queue: Defines the queue name to send the message to -->
    <!-- connection: For AMQP connections, the connection name must match the connection attribute in the queue_topology.xml file. Otherwise, the connection name must be db -->
    <!-- consumerInstance: The Magento class name that consumes the message -->
    <consumer name="klizer_queue_example" queue="klizer_queue_example" connection="amqp" handler="Klizer\QueueExample\Model\MessageQueues\QueueExample\Consumer::processMessage"/>
</config>

Step 6: Create a Consumer File

The Consumer class processes the message received from the queue. This is where the message is decoded from JSON and processed further (e.g., stored in the database, logged, etc.)

Path: app/code/Klizer/QueueExample/Model/MessageQueues/QueueExample/Consumer.php

<?php
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
 */

namespace Klizer\QueueExample\Model\MessageQueues\QueueExample;

use Magento\Framework\MessageQueue\ConsumerInterface;
use Magento\Framework\App\ResourceConnection;

class Consumer
{
    const KLIZER_SERVICE_TABLE = 'klizer_services';

    const SERVICE_NAME = 'services_name';

    const DESCRIPTION = 'description';    

    /**
     * @var \Magento\Framework\App\ResourceConnection
     */    
    private $resourceConnection;

    public function __construct(
        ResourceConnection $resourceConnection
    ){
        $this->resourceConnection = $resourceConnection;
    }

    public function processMessage(string $queueData)
    {
        $queueData = json_decode($queueData);
        $connection  = $this->resourceConnection->getConnection();
        $tableName = $connection->getTableName(self::KLIZER_SERVICE_TABLE);
        $data = [
            self::SERVICE_NAME => $servicesName,
            self::DESCRIPTION => $description,
        ];
      $connection->insert(self::KLIZER_SERVICE_TABLE, $data);
    }
}

Step 7: Create queue_publisher.xml

Use: Defines the exchange where a topic is published.

Path: app/code/Klizer/QueueExample/etc/queue_publisher.xml

<?xml version="1.0"?>
<!--
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
*/
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Queue:etc/publisher.xsd">
    <publisher topic="klizer.queue.example">
        <connection name="amqp" exchange="klizer.queue.example" />
    </publisher>
</config>

Step 8: Create Publisher

You need to define a publisher class responsible for sending messages to the RabbitMQ queue.

Path: app/code/Klizer/QueueExample/Model/MessageQueues/QueueExample/Publisher.php

<?php
/**
 * Klizer
 *
 * @category  Klizer
 * @package   Klizer_QueueExample
 * @copyright Copyright © 2024 Klizer. All rights reserved.
 * @author    Klizer - info@klizer.com
 */

namespace  Klizer\QueueExample\Model\MessageQueues\QueueExample;

use Magento\Framework\MessageQueue\PublisherInterface;
use Magento\Framework\App\ResourceConnection;

class Publisher
{
    /**
     * @var \Magento\Framework\MessageQueue\PublisherInterface
     */    
    private $publisher;

    public function __construct(
        PublisherInterface $publisher
    ){
        $this->publisher = $publisher;
    }

    public function publishMessage(string $queueData)
    {      
        $this->publisher->publish('klizer.queue.example',$queueData);
    }
}

In the above code:

• PublisherInterface is used to send messages to a specific queue or exchange.
• The method publishMessage() takes a JSON-encoded string and sends it to the queue identified by the topic name klizer.queue.example.

Publish Data Using the Publisher (Insert Data Using RabbitMQ In Magento 2)

Now you can use the publisher to send data to the queue. Here’s how you can publish a message:

Example:

use Klizer\QueueExample\Model\MessageQueues\QueueExample\Publisher as Publisher;

/**
 * @var \Klizer\QueueExample\Model\MessageQueues\QueueExample\Publisher
 */
private $publisher;

public function __construct(
    Publisher $publisher
) {
    $this->publisher = $publisher;
}

$servicesName = 'Adobe Commerce';

$description = 'Adobe Commerce gives you total control over your online store, covering design, development, deployment, and management.';

$_data = array(
    'servicesName' => $servicesName,
    'description' => $description
);

$this->publisher->publishMessage(
    json_encode($_data)
);

Breakdown

• Publisher: Handles sending the message to RabbitMQ.
• Consumer: Consumes and processes the message.
• Queue: Defines the queue and exchange in the RabbitMQ configuration.
• Data: Example data, such as service details, is published as JSON.

Conclusion 

Integrating RabbitMQ with Magento 2 allows you to simplify your eCommerce business operations by enabling asynchronous messaging, decoupling components, and managing high-traffic loads in simple techniques. Implementing the above message queues can be helpful for your platform as it can handle background tasks efficiently, ensuring a seamless experience for your customers while improving the overall performance and scalability of your business.

At Klizer, we specialize in setting up and optimizing Magento 2 platforms with RabbitMQ integration. Our team of Magento development experts can help you implement a robust messaging system tailored to your business needs. If you’re seeking help or need assistance from experts, feel free to reach out to us.