Add Kafka Topic Health Check in ASP.NET Core
In this article, we shall check how to implement the Apache Kafka Topic Health Check in ASP.NET Core API or MVC-based applications.
Today in this article, we will cover below aspects,
In our previous article on how to get started with Kafka in ASP.NET Core, we learned how to use Kafka using simple Consumer and Producer examples in ASP.NET Core 3.1 or 6.0 API.
In the above article, we created an API that acts as a traditional service listening to business events continuously as produced by another Kafka event producer.
- Implementing a health check might be mandatory in many stages of the application development life cycle.
- It helps you to keep track of the health of numerous resources, databases, and services that your business application depends on in real time.
- This enables you to quickly troubleshoot and resolve technical or environmental issues.
Health check services and middleware in ASP.Net core allow us to validate APIs by allowing us to create health check routes.
Kafka Health Check can be implemented by multiple approaches. You can verify Kafka topic health using either,
- Producing the event to a given topic and verifying the status Or
- Consuming the event message from a given topic (provided you have the right consumer group or consuming messages will not affect any existing consumption)
Kafka Topic Health Check – Getting started
If you already have an API, then please follow below two steps below,
Create Kafka health checks using the IHealthCheck interface
Creating Kafka health checks using the IHealthCheck interface is simple and it is explained in detail below
Step1 – Implement Kafka IHealthCheck interface for Kafka Producer
Here, please implement the IHealthCheck interface and override the CheckHealthAsync method. IHealthCheck check is readily available in the below namespace from Microsoft,
below we are defining KafkaHealthCheck check using ProducerConfig class. This class is available to use from NuGet packages as explained in the below article,
We shall be using Producer capability where we will be pushing the messages to the Kafka topic and verifying Kafka topics access the messages command.
public class KafkaHealthCheck : IHealthCheck
{
private readonly ProducerConfig _producerConfig;
private string _topicName;
public KafkaHealthCheck(ProducerConfig producerConfig, IConfiguration configuration)
{
_producerConfig = producerConfig;
_topicName = configuration.GetValue<string>("KafkaConsumer:HealthTopicName");
}
..
..}
Above ProducerConfig is configured at the Startp.cs and then dependency injected by the API pipeline which is already explained in the below article Getting Started with Kafka in ASP.NET Core
Below is how we are configuring the ProducerBuilder,
using var p = new ProducerBuilder<Null, string>(_producerConfig).Build();
below is the CheckHealthAsync which is the IHealthCheck interface method implemented for Kafka producer connection.
Below is the implementation of the method,
public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default)
{
try
{
using var p = new ProducerBuilder<Null, string>(_producerConfig).Build();
var result = await p.ProduceAsync(_topicName, new Message<Null, string> { Value = $"Kafka is healthy on {DateTime.UtcNow}" }, cancellationToken);
if (result.Status == PersistenceStatus.NotPersisted)
{
return new HealthCheckResult(context.Registration.FailureStatus, description: $"health check for kafka topic failed.");
}
return HealthCheckResult.Healthy();
}
catch (Exception ex)
{
return new HealthCheckResult(context.Registration.FailureStatus, exception: ex);
}
}
HealthCheckResult lets you return health as
Healthy,Degraded, orUnhealthystatuses.
In the above method, for any connection failure, you shall get an exception that can be used to return the failure status of the method.
Step2 – Register Kafka health check services in API
In Startup.ConfigureServices please add the HealthChecks as below,
Below is a sample Configure method,
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UseHttpsRedirection();
app.UseHealthChecks("/healthcheck");
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
}
Let’s execute the route “/health check” and we shall get the Kafka Health check success as below,
You shall be able to verify Healthy, Degraded, or Unhealthy status accordingly.
Step 3 – Implement Kafka IHealthCheck interface for Kafka Consumer
If you need to verify the Consumer side of Kafka topics, then the above approach can be extended by adding the producer and consumer together. In that way, you can verify the messages for the producing and consuming sides simultaneously.
For Consumer side implementation – Plz get it here.
Useful References:
Do you have any comments or ideas or any better suggestions to share?
Please sound off your comments below.
Happy Coding !!
Summary
A health check is critical in multiple aspects and allows you to check the health of various resources, databases, and services that applications are critically dependent on. Today in this article, we looked at a simple approach of enabling Health Check for Kafka Topics (using both the consumer and producer ) in the .NET application.
Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published(2024) best practices and guidelines for software design and development.