Posts

A common discussion As a Customer Success Technical Architect, I am often asked how to manage Kafka resources. The question usually comes up when a cluster is growing, a platform team is reviewing capacity, or an architecture needs to become more efficient: How can we reduce Kafka’s footprint without weakening the guarantees our applications need? Kafka can look deceptively light at the beginning. Create a topic, send a few records, and the cluster seems to have plenty of room. The picture changes as the platform grows. More topics bring more partitions. More replicas bring more copies of the data. More consumers can bring more fetch traffic and consumer-group coordination. ...

Kafka records have timestamps, and most of the time we barely think about them. That changes quickly when a producer sends a timestamp in nanoseconds instead of milliseconds. A record can suddenly look like it came from hundreds of years in the future. Kafka may accept it, and the original mistake can stay hidden until it affects retention, log segments, or downstream consumers. KIP-937 addresses this problem directly. It separates timestamps in the past from timestamps in the future, so operators can allow legitimate historical data without also accepting dates that are clearly implausible. ...

A common question when designing a Kafka cluster is: how many partitions can one broker support? There is no single answer for every cluster. Kafka topics use partitions so producers and consumers can work in parallel. But every partition also creates work and state on the brokers that host it. The practical limit depends on broker resources, replication factor, Kafka version, and operational needs. Treat partition counts as an operational capacity limit, not only as a logical design choice. ...

PyconEs 2024 in Vigo PyConES is the annual Python conference in Spain, and it is organized by the Python Spain association. The Python community in Vigo is quite active and so, they were invited to host PyConES 2024. Just to mention too, Python Vigo was a pillar to the creation of VigoTech Alliance, the meta community of tech communities in Vigo. Keeping the long story, Confluent was an sponsor (and a colleague had a talk), so I had no excuses to not attend it. If you do not want to hear more, the TL;DR is that I was impressed by the number of people, the quality of the talks and the organization. Congrats to everyone involved! ...

Article in Confluent Blog My first article on Confluent Blog How to Use Confluent for Kubernetes to Manage Resources Outside of Kubernetes talks about how to manage your Confluent resources like RBAC, topics, schemas and connectors using Kubernetes in Confluent components living outside Kubernetes. The article is complemented with a full demo and I hope it can be an inspiration if you are familiar with Kubernetes and uses Confluent components in VM machines or similar.

Why Travis CI is blocking this blog to be generated, why? I do not know, I have credits for it, but it cannot be used, so I am forced to tick a task in my list and move to GitHub actions. Disabling Travis CI Removing GitHub <-> Travis CI integration Go to https://github.com/settings/installations Search “Travis CI” and click Configure Scroll down and click in the “Uninstall” button Go to https://github.com/settings/apps/authorizations Search “Travis CI” and click Revoke Deleting Travis CI According to this forum question, there is no UI option. Send a mail to support@travis-ci.com requesting the account deletion. Add GitHub Actions That is the easiest part as Hugo created a nice tutorial. I followed it and that is the reason why you are able to read this blog post.

In 2019, I gave a talk at Netcentric Summit about Junit5 migration and its advantages. You can check the content here: Video: Netcentric | JUnit 5, mockito 2 and AEM Mocks, is it true for you? Blog post: JUnit 5: leveraging new features for testing After 3 years, I wanted to revisit the procedure to migrate from junit4 to junit5, so let’s do it! The result was this PR. Steps 1. Dependencies 1.2 Main pom.xml In the main pom.xml, replace ...

I was doubting about writing this post because AWS is evolving really fast and this post will be probably old very fast. So, I am adding the AWS documentation, so, please take your time to check the official documentation to confirm my findings (and if possible, write me a message so I can update it if needed). SQS AWS SQS is a simple 1:1 queue, so it delivers messages from A to B. ...

If you are creating a cloud of microservices or are in an environment where you have to generate multiple maven projects or modules, it is very useful to create archetypes to handle these new modules/projects creation. What is an archetype? A Maven archetype is a templated project. Maven offers several archetypes that allow you to create new maven modules ready to run in few seconds. How to use a generic archetype? Archetype usage is quite simple. If you want to generate an archetype based on the default list, just run: ...

I am starting my journey with kubernetes (k8s) and I want to share the tutorials and blog posts that I read. In that way I can find them easily too 😬. Kubernetes basics: A Beginner’s Guide to Kubernetes: a very complete guide about the kubernetes basics and how they work internally. An introduction to Kubernetes.: defines kubernetes, pod, deployment, service, ingress, job, kubernetes control plane and when to NOT use k8s. Kubernetes tools: Developer Tooling for Kubernetes in 2021: Helm, Kustomize, and Skaffold. Important point here: kustomize is part of kubectl calling kubectl -k, so it will, possibly, become the standard. How To Manage Your Kubernetes Configurations with Kustomize Expose Your App to The Internet Using Ingress Controller kaniko - Build Images In Kubernetes This list will grow, but if you found any error or want to help me to improve this article, please comment the pull request