Confluent

Kafka records have timestamps, and most of the time we barely think about them. That changes quickly when a producer sends nanoseconds instead of milliseconds. Suddenly, a record can appear to come from hundreds of years in the future. Kafka may accept it, while the original mistake stays hidden until it starts affecting retention, log segments, or downstream consumers. KIP-937 addresses this problem directly. It separates timestamps in the past from timestamps in the future, so operators can allow legitimate historical data without also accepting dates that are clearly implausible. ...

A common question when designing a Kafka cluster is: how many partitions can a broker support? There is no single answer that applies to every cluster. Kafka topics are divided into partitions so that producers and consumers can work in parallel, but every partition also creates work and state for the brokers that host it. The practical limit depends on the broker resources, replication factor, Kafka version, and operational requirements, so partition counts should be treated as an operational capacity limit, not only as a logical design choice. ...

Article in Confluent Blog My first article on Confluent Blog How to Use Confluent for Kubernetes to Manage Resources Outside of Kubernetes talks about how to manage your Confluent resources like RBAC, topics, schemas and connectors using Kubernetes in Confluent components living outside Kubernetes. The article is complemented with a full demo and I hope it can be an inspiration if you are familiar with Kubernetes and uses Confluent components in VM machines or similar.