Lamport's Logical Clock - Ordering Events in a Distributed System

0 comments

Basically a distributed system is one in which the components or processes or nodes that comprise a system is distributed across nodes or sometimes even across geographies. But these systems need to communicate with each other to accomplish something meaningful. The most efficient, scalable and proven way of making these distributed systems communicate is via passing messages, sometimes even having a messaging middleware (like Kafka for example.,) as a central component.

Now you can imagine when there is a very big carousel of such applications distributed across several nodes, where each communicate via passing messages, we need to somehow find a way to guarantee message ordering. Ok, yo we can rely on the system clock of the node? May be, but may be not. Here is why - System clocks struggle from Clock Skew and Clock Drift. On a distributed system, you cannot have something that acts like a global clock as that would one massively bring down scalability and increase your single point of failure. So a finesse solution would be to have something like a logical clock that each node in the application tracks and knows how to adapt it's clock against it's own message evolution and for the messages that it received from other nodes. One such algorithm is the so called Lamport Logical Clock

I will try to simplify with my explanation on how this algorithm works.

For the sake of simplicity, let us assume that in our world of distributed systems, only 2 nodes exist. I call it Node N1 and Node N2. These Nodes care about the following Events: 1. Events that are generated internally in the Nodes - I call them Node Event or simply NE 2. Events that are sent from one Node to the other - I call them Send Event or simply SE 3. Events that are received on a Node from the other Node - I call them Receive Event or simply RE 4. All nodes start with a Zero counter

Lamport's logical clock states that each Node should have some form of incremental value (could be timestamp or an integer) that can be associated with an event and could be compared against another value. So in this case, let us assume that we just have a simple integer counter for our Nodes and this counter will be incremented for every event that happens (be it NE, SE or RE), but how it is incremented depends on the following conditions: For any NE, the counter will be incremented and the new Event will have the incremented counter value For any SE or RE, the following has to happen: 1. The sender Node need to send the Event with the current counter value that the Event which is being sent has 2. The receiver Node, upon reception of the message, should then check for the counter value and should make sure it satisfies the following conditions: 2.1 The time of reception should be greater that the time of send 2.2 The receiver node should check against it's very own logical counter, and use the formula below to update the counter value in the newly received message new counter value = max(receive event counter++, local counter)

So what happens at 2.2 is that as soon as a Node receives an Event, it immediately increments the counter value which is in the event and then checks / compares against its local counter value and whichever is the maximum, will then be this event's new counter value. The global counter in this node is then updated with this latest counter value.

So there you have it, without any pictorial representation, we have the Lamport Logical Clock unbundled!

Deeply dissapointed with Travis CI and the way OSS is treated

0 comments

I have been a happy user of the Travis CI free usage over the last couple of years. But one announcement recently made me deeply worried:

Have a look here at their announcement. Yes, Travis CI will no longer be free for OSS projects. There is no point in blaming Travis CI for this but rather on those idios who abused the free usage with crypto minites and tor nodes. These bastards completely broke the trust Travis CI free tier gave for OSS projects. I loved their UI and the way the builds are executed. I will no longer have the chance to use them. There is a free tier still available but they are limited to 1000 minutes and this would mean that I can hardly do roughly 120 to 160 builds and I'm not sure if this is spread over a year. But it is definitely not enough! Once they are exhausted, I probably need to beg for more. This is no fun!

Mark & run integration unit tests with SBT and Scala

0 comments

It is a good practice to write integration unit tests for your services and integrate them in your CI workflow. Such integration tests boosts the confidence of your application quality especially when it needs to be deployed in an environment where it has to talk to numerous other external services. In this regard, I wanted to write such integration tests for one of my scala project which uses SBT as a build tool.

I will just list down the steps that are needed to get this up and running!

First, in your build.sbt define the configuration and settings that can help sbt to differentiate between your integration unit tests and normal unit tests.

// Command to run integration tests, so here to run the integration tests we use -> sbt integration-test
addCommandAlias("integration-test", "Integration/testOnly -- -n integrationTest")

lazy val IntegrationTest = config("integration").extend(Test)

lazy val root = Project(base = file("."))
  .configs(IntegrationTest)
  .settings(
    IntegrationTest / parallelExecution := false, // We do not want Integration tests to execute parallely!
    Test / testOptions += Tests.Argument(TestFrameworks.ScalaTest, "-l", "integrationTest"), // Exclue the Inegration tests from the normal unit tests
    // Include integration tests, by nullifying the above option
    IntegrationTest / testOptions := Seq.empty,
  )
  .settings(
    // Enable integration tests
    inConfig(IntegrationTest)(Defaults.testTasks)
  )
We need to do a few more steps before we can get the whole thing up and running. We need to define an object that extends the Tag for your testing framework. So in my case I use scalatest and for me it would look like:

import org.scalatest.Tag
object IntegrationTest extends Tag("integrationTest")

After this, we can now start writing our test classes and tag them as IntegrationTest:

  test("integration testing with sbt", IntegrationTest) {
    Future {
      println("ALLES OKAY *************************************** ")
      assert(true)
    }
  }

Now you can run your integration tests separately for your project as below:

sbt integration-test

Once more a blow, a nasty one this time

0 comments

I could not say this more than what the title says. I had a very severe blow to my already damanged left knee. Here and here is a bit of history It was on the 28th of September where I went for my first ever defensive sport training called "VingTsun". Inspired by my Phsio friend who is a beginner, I decided to give this training a try. I indeed watched few videos of it before commiting myself to joining him for the training session which starts around 20:00 Hours near to where I live.

Convinced that if during the training, should I need to use my legs a lot, I would talk to the trainer and avoid such moves, I happily started my session. Well guess, in the first 10 minutes where we were just warming up, I inflicted a severe blow to my left knee. What happened was, we were asked to run, run, stop, jump, run. Run was Okay, Run was Okay, Stop was Okay, but when I jumped (I guess it was the third time of my jump), and landed, my left kneww buckled and I fell down with such an excruciating pain. The pain was so intense that I could not breathe anymore and I was gasping for air.

My Phsio friend tried to help out by pressing his hand against my knee, but that did not help. For the next 10 minutes I was just shouting out loud without being able to bear the pain. I got some ice applied and I was just moved over to the corner of the room. Since my home is a few kilometers away and I was not in a position to travel back alone, I waited for the session to finish. My physio friend toed my on my bike back home.

Fast forward a week, I'm back with my crutches. I think it is time for me to think about getting this damn thing operated. The pain was there for almost a week and I still have the effect in my knee from the impact.