(888) 685-3101 ext. 2

Introduction:

Kafka is a scalable pub/sub system, primarily used to collect & analyze large volumes of data. It enables us to pass messages from one end-point to another. Kafka messages are persisted on the disk and replicated among the cluster to prevent data loss. Kafka is constructed on top of the ZooKeeper synchronization service.

Benefits:

  1. Reliability
  2. Scalability
  3. Durability
  4. Performance

Use Case:

Send and Receive message from Kafka messaging system by integrating with the Mulesoft platform.

Please follow the step by step installation procedure of Kafka Installation.

Mule Flow (Publish Message):

Mule flow publish message

Step -1: Configure the HTTP Listener by giving it a hostname, port number, and path. Along with this, specify allowed methods (Optional) in the Advanced tab of the HTTP connector.

Step-2: Drag & Drop the Logger component to log the message that was consumed by Kafka.

Step-3: Kafka comes as a custom connector needed to get it from Exchange. So, here Drag & Drop the Kafka Publish Message from Exchange.

kafka publish message

From above:

Topic : Message to send.

Key : Belongs to message that is going to be sent.

Mule Flow (Consume Message):

Step-4: Drag & Drop the Kafka Message Consumer connector as the source from the palette.

kafka message consumer

From above:

Topic : Message to receive.

Group id : Unique string that identifies consumer group that this consumer belongs to.

Step-5: Drag & Drop the Logger component to log the consumed message.

Final Config.xml:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:kafka="http://www.mulesoft.org/schema/mule/kafka" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns="http://www.mulesoft.org/schema/mule/core"
	xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/kafka http://www.mulesoft.org/schema/mule/kafka/current/mule-kafka.xsd">
	<kafka:kafka-producer-config name="Apache_Kafka_Producer_configuration" doc:name="Apache Kafka Producer configuration" doc:id="0289a213-cdf8-49ab-a13b-59af6cb25f97" >
		<kafka:basic-kafka-producer-connection bootstrapServers="localhost:9092" />
	</kafka:kafka-producer-config>
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="7700a54a-38e2-4385-a195-6a539d201b37" >
		<http:listener-connection host="0.0.0.0" port="8082" />
	</http:listener-config>
	<kafka:kafka-consumer-config name="Apache_Kafka_Consumer_configuration" doc:name="Apache Kafka Consumer configuration" doc:id="6e24bf87-b86e-4319-8ec2-624f68b245b5" >
		<kafka:basic-kafka-consumer-connection groupId="1" bootstrapServers="localhost:9092" />
	</kafka:kafka-consumer-config>
	<flow name="Kafka-Publish" doc:id="47de8881-580b-41d2-af4f-4f505480d9ec" >
		<http:listener doc:name="Listener" doc:id="173de33a-4b7d-4fd1-951c-a153dde6b52a" config-ref="HTTP_Listener_config" path="/kafka" allowedMethods="POST"/>
		<logger level="INFO" doc:name="Logger" doc:id="c85d216f-b005-4663-9309-28864a818da6" message="#[message.payload]"/>
		<kafka:producer doc:name="Publish Message" doc:id="e5457b47-7dcd-42bc-b83d-d6922dec0641" config-ref="Apache_Kafka_Producer_configuration" topic="muleesb" key="#[now()]">
			<kafka:message ><![CDATA[#[message.payload]]]></kafka:message>
		</kafka:producer>
	</flow>
	<flow name="Kafka-Consumer" doc:id="c0e6fd8a-0bcd-4ccb-91b6-0e97804d5607" >
		<kafka:consumer doc:name="Message Consumer" doc:id="660a5e95-a897-4cdb-a971-10df13d9d4be" topic="muleesb" config-ref="Apache_Kafka_Consumer_configuration"/>
		<logger level="INFO" doc:name="Logger" doc:id="86114247-dce0-4626-8278-81450bc928b6" message="#[payload]"/>
	</flow>
</mule>

Sample Input & Output:

sample input and output

H@ppy Le@rninG…@@

Conclusion

We have successfully integrated a Kafka connector to send an automatic email notification to the customers in one of the Order Management System POC. For additional details, please submit your questions in the below form. We are happy to assist you.

Murali Thuraka | Software Engineer – L2
Murali Thuraka is a Software Engineer on the MuleSoft and EBX team at XTIVIA, Inc. He is an experienced engineer with a demonstrated history of working in the IT and services industry. He’s a strong engineering professional with a Bachelor of Technology focused in Mathematics from ZPH-School and is skilled in Mulesoft, Core Java, Java, XML, HTML, and XML Schema Definition (XSD). Read More from Murali Thuraka

Prudhvi Vobbilireddy | Software Engineeer-L1
Starting in 2017, Prudhvi Vobbilireddy has been working as a talented software engineer on XTIVIA, Inc.’s MuleSoft & EBX team. Prudhvi has more than four years of experience with MuleSoft technologies (Anypoint platform, MuleSoft Runtime Environment, and Management Console) and has multiple certifications in MuleSoft, Oracle, and Splunk technologies. His background in SOA-based technology and implementation has allowed him to understand complex software problems and provide concise solutions. Read More from Prudhvi Vobbilireddy

Sankar Kenguva | Software Engineeer-L1
Sankar Kenguva is a Software Engineer-L1 on the MuleSoft & EBX team at XTIVIA, Inc. As an experienced software developer and certified MuleSoft engineer, Sankar creates optimal solutions for XTIVIA clients and delivers significant results for his team in Enterprise Information Management (EIM). Read More from Sankar Kenguva

Share This