Smily Engineering Blog
Karol Galanciak
by Karol Galanciak
7M ago
We’ve finally launched Smily (formerly BookingSync) Engineering Blog which I’m going to primarily focus on instead of this blog here. If you enjoyed the content of my personal blog, make sure to check Smily blog as it’s going to be even better ..read more
Visit website
An alternative approach to custom partition assignment strategy for Kafka consumers with Karafka
Karol Galanciak
by Karol Galanciak
1y ago
Recently, in BookingSync, we were performing a migration from Karafka 1.4 to 2.0, which we use for communication with Kafka. One of the great features available in version 1.4 was a custom partition assignment strategy for consumers. It was particularly useful for us as we’ve had some topics that had a way higher throughput than the other ones, so just a round-robin strategy with even distribution of topics for consumers was not a suitable choice as we needed dedicated consumers for these specific topics/partitions. Unfortunately, custom partition assignment strategy for consumers is no longer ..read more
Visit website
The inherent unreliability of after_commit callbacks and most service objects’ implementation
Karol Galanciak
by Karol Galanciak
1y ago
Service objects and/or after_commit callbacks are ubiquitous in most real-world Rails applications. Whether it’s a good idea or not (ActiveRecord callbacks - I’m looking at you) is a different story, but one thing that is notoriously overlooked in the application design is reliability. And yes, the service objects are equally bad as after_commit callbacks in that regard. Let’s explore the concept of reliability and also some solutions that can help solve the problem. Using transactions and reliability Typically, after_commit callbacks are used for executing side effects after the transaction ..read more
Visit website
The Story of a Critical Issue With Kafka
Karol Galanciak
by Karol Galanciak
2y ago
Recently, I’ve had an unfortunate opportunity to deal with a very unexpected issue with Kafka that had quite terrible consequences. The exact origin of the issue is yet to be discovered; nevertheless, the process leading to the final solution to the problem and the fix itself were interesting enough that I decided to write a quick blog post about it as it might be potentially valuable to someone who also encounters a similar problem. How the issue was diagnosed Suddenly, after one of the deployments (nothing related to any change around Kafka or karafka/ruby-kafka gems that we use in the app ..read more
Visit website
Multiple databases in a single query in your Rails apps - Postgres Foreign Data Wrappers to the rescue
Karol Galanciak
by Karol Galanciak
2y ago
Imagine that you are building a separate application for your e-commerce system dedicated to business intelligence. In other words, you want to calculate some stats for the orders. So you are going to create some new model, like OrderStat(s), and have a separate Postgres database for a new app. Sounds trivial so far. However, how are you going to get the data from the actual Orders? One way of doing that would be to do some data liberation and stream all Orders (or events related to orders) into Kafka and let the consumers get the data from Kafka. If that’s ever your plan to have Kafka in your ..read more
Visit website
Kafka for Rubyists Mini Course: Advanced Karafka
Karol Galanciak
by Karol Galanciak
3y ago
This is the fourth and also the last part of the Kafka For Rubyists mini course. If you are interested in other parts of the course, please check them out here. 4. Advanced Karafka Video Slides ..read more
Visit website
Kafka for Rubyists Mini Course: Advanced Kafka
Karol Galanciak
by Karol Galanciak
3y ago
This is the third part of the Kafka For Rubyists mini course. If you are interested in other parts of the course, please check them out here. 2. Advanced Kafka Video Slides ..read more
Visit website
Kafka for Rubyists Mini Course: Consuming and Performance
Karol Galanciak
by Karol Galanciak
3y ago
This is the second part of the Kafka For Rubyists mini course. If you are interested in other parts of the course, please check them out here. 2. Consuming and Performance Video Slides ..read more
Visit website
Kafka for Rubyists Mini Course: Fundamentals
Karol Galanciak
by Karol Galanciak
3y ago
This is the first part of the Kafka For Rubyists. If you are interested in other parts of the course, please check them out here. 1. Fundamentals 1.1 Intro to Kafka Video Slides 1.2. Modelling topics and publishing messages Video Slides 1.3. Karafka in practice: 1.3.1 Producers Video 1.3.2 Consumers Video 1.4. Monitoring Video Slides ..read more
Visit website
Kafka for Rubyists Mini Course
Karol Galanciak
by Karol Galanciak
3y ago
I’ve been recently planning to record a couple of tutorials for my team in BookingSync, focusing mostly on monitoring and some other more advanced aspects of Kafka and Karafka. But if I was already recording something, then why not make it available publicly so that more developers could benefit from it? So, in the end, I decided to make a bit more effort and recorded this mini-course, which will be available for everyone, for free ;). This course consists of 5 blocks (including intro): 0. Intro to course Intro 1. Fundamentals 1.1 Intro to Kafka Video Slides 1.2. Modelling topics and pub ..read more
Visit website

Follow Karol Galanciak on FeedSpot

Continue with Google
Continue with Apple
OR