2023
Spark - Needle in a haystack story
In this short blog post, I will share an example of how executor logs helped us in narrowing down a malformed record that was causing our code to throw java.lang.ArithmeticException long overflow exception
2021
Mock External Dependencies via Traits
In this short blog post, I will give an example wherein we will see how with the help of Scala trait’s we can mock external dependencies.
Localstack & Terraform
In this blog post, we are going to see an example of how we can use the localstack framework for testing terraform deployments.
2019
Spark Patterns - FlatMapGroups
In this blog post, I am going to explain you with an example on how we can use the FlatMapGroups api for implementing complex logic against grouped datasets.
2018
Hive & Ldap Authentication
In this post, I will cover the steps required to properly configure Ldap & corresponding group level authentication in hive.
Running Spark App In Standalone Cluster Mode
In this blog post, I will explain the process of running Spark application against locally running standlaone cluster.
2017
Working with schema in SparkSQL
In this blog post, we will see how to apply schema to SparkSQL DataFrames. We will also see, how to use Scala’s implicits for converting DataFrame into strongly typed entities.
SparkSQL Getting Started
In this blog post, I am going to explain you the steps required for configuring Spark in your machine. I will also present simple SparkSQL program which runs SQL query against sample csv file.
Spark Recipes
If we ignore the complexities of running spark applications then getting up-to speed with spark programming api is relatively straight forward. However like any other programming api, spark too contains some elements that aren’t that obvious to figure out. In this post, I will share some not so obvious things about spark programming api.
Hive - How to easily test custom auth Jar?
In this post, I will share the steps you can take to easily test your custom hive authentication implementation.
Using Symbolic Links to Manage Libraries
This post is quick tutorial on how you can easily manage & switch between different versions of programming libraries that you have configured in your machine with the help of symbolic links.
Mixin in Scala
In this blog post, we will look into how Scala provides support for mixins via traits. Mixin allow classes to provide functionalities to other classes without making the other classes inherit from them.
SSH Config
Unhandled & Dead Messages in Akka
In this blog post, we are going to look into two special cases of message delivery & its handling by the akka framework. First case involves sending invalid message & second case involves sending messages to dead actors.
Scala Topics - Actor Based Programming
In this blog post, I am going to use Akka’s Actor framework for implementing code for scoring Bowling game. Before getting into the code, I will provide quick introduction to Actors & components involved in actor based programming.
Setting date & time in Linux VM
In this blog post, I will provide you steps for setting date & time in Linux VM through command prompt.
Scala Topics - Covariance & Contravariance
In this blog post, I am going to explain you the difference between covariance & contra-variance. If you are not familiar with these terms then let me tell you that its related to the way type parameters are handled(more on this) when defining generic types or methods.
2016
Advent Of Code 2016
I recently came across this site called Advent Of Code which lists set of problems in increasing order of complexity. If you are like me, trying to learn new programming language(in my case it’s Scala), then solving handful of such problems will greatly expedite your learning process.
Scala Topics - Case Classes
In this blog post, I am going to cover Scala’s Case class functionality. Scala being functional programming language, introduces new programming constructs like Case classes, traits & other features to support & enhance functional programming experience.
Postgres and Json - Part2
In this blog post, we are going to explore some of the json related operators & functions specific to filtering & processing of json based data.
Postgres and Json - Part1
In this blog post, we are going to look into the various json related functions available in postgres database specific to json creation.
Logback for Beginners
In this blog post, I am going to explain two key concepts of logging - log levels & logger hierarchy using logback logging framework.
AOP in Java
In this blog post, I am going to use an open source library called jcabi-aspects for implementing loggable aspect.
Kafka - Getting Started
In this blog post, I am going to outline the steps required for setting up Kafka in your local development machine.
Working with MongoDB using F# & C#
Postgres - BackUp & Restore
In this blog post, I am going to share with you all the commands required for taking database back-up & restoring database from those back-up files.
RxJava - SubscribeOn & ObserveOn
In this blog post we will cover the two most important aspect of RxJava programming which is configuring observeOn & subscribeOn listeners.
Why Blog?
I have recently joined the local Toastmasters club in my office. The idea behind the Toastmasters club is to promote & improve public speaking by encouraging participants to give speeches either prepared lengthy speech lasting about 5 to 7 minutes or do an impromptu speech for about 2 minutes on any random topic. I can say with confidence that I have experienced drastic change in myself. I no longer shiver with fear of giving speech by facing large group of audience. Now this blog post is not going to be about the merits of Toastmasters instead I am going to use this post as placeholder for my next speech which is going to be on public writing aka Blogging.
RxJava - Getting Started
2015
MEANT Stack Part4 - Front End Implementation - 2
Table of Contents
This is the last part of the four part blog series on building web applications using MEAN stack, TypeScript and Visual Studio Code. In the previous three parts we have covered the following things :
MEANT Stack Part3 - Front End Implementation - 1
In this and the next module, we will concentrate on building front-end of our application using AngularJs, TypeScript & Bootstrap.
MEANT Stack Part2 - Node & Mongoose SetUp
Table of Contents
In the previous post, we went through the steps of installing necessary software’s & modules required for building our application. In this post, we are going to concentrate on our application backend i.e. setting up node as as web server & mongoose related codebase for performing CRUD activities in the underlying mongo database.
MEANT Stack - Part1
Table of Contents
In this multi-part blog series, I will walk you through on how to build a small web application using Nodejs, MongoDb, TypeScript & AngularJs. We will be building a small web application for creating and registering teams. A team will have team members and a unique team name. The application will have two screens. First one used for creating a new team and the other one used for listing all the registered teams. In this post, we will concentrate on installing & setting up of required software’s. Let’s get started.
Going Functional - Symbol Graph in F#
In the previous post, we have seen the implementation of undirected graph data structure in F#. In this post, we will make use of the Graph data structure to implement the Symbol Graph data structure. You can read more about Symbol Graph data structure here. Given below is the java based implementation of Symbol Graph data structure. The implementation if taken from Algorithms 4th Edition by Robert Sedgewick and Kevin Wayne.
Going Functional - Breadth & Depth First Search in F#
In this blog post, I will provide the depth and breadth first traversal implementation in F#. But before that lets look at the Java based implementation of these traversal mechanisms. The code is taken from Algorithms 4th Edition by Robert Sedgewick and Kevin Wayne. You can find the complete code here : http://algs4.cs.princeton.edu/40graphs/
Going Functional - Priority Queues F#
In this blog post, I will provide the F# implementation for max priority queue. First the java based implementation taken from Algorithms 4th Edition by Robert Sedgewick and Kevin Wayne. You can find the complete priority queue implementation here : http://algs4.cs.princeton.edu/24pq/MaxPQ.java.html
Going Functional - Merge & Quick Sort in F#
Continuing our functional journey, in this post I will first present the Java based implementation of merge sort followed by F# based implementation. Finally we will repeat the same steps for QuickSort algorithm. Java based implementation of the sorting algorithms is taken from Algorithms 4th Edition by Robert Sedgewick and Kevin Wayne.
Going Functional - Elementry Sorting Algorithms in F#
In this blog post, I will provide F# implementation for Selection & Insertion sorting algorithms. First the C#/Java based implementation of these algorithms taken from Algorithms 4th Edition by Robert Sedgewick & Kevin Wayne.
Going Functional - Stack Implementation in F#
In this blog post, I will port linked list based Stack ADT code in C# to F#. Given below is the C# implementation. It’s inspired from the Java based implementation provided in the Algorithms 4th Edition book by Robert Sedgewick and Kevin Wayne. The Stack ADT class is called LLStack because .Net framework itself contains Stack data structure.
Is Screen Saver running or not?
Although its rare but at times while building desktop based applications, we might come across situation wherein we are asked to handle “Screen Saver” operation i.e. perform some logic based on whether the “Screen Saver” is on of off. I found the following code snippet on internet almost two years back. I would like to thank the original author of the code. It’s been long time and its not possible for me to search for the original author of the code.
Async Web Api Performance
In one of my recent WPF project, we made extensive use of async-await pattern. Async-await pattern greatly simplified the call-back and continuation based code required for keeping the UI responsive. In WPF, the pattern for implementing async-await is to invoke the IO/CPU intensive code in a background thread and attach the continuation logic on the main UI thread. In WPF, since we have a dedicated UI thread that controls all of the UI elements, using async-await is really helpful in keeping the UI responsive without having to write complicated call-back based code.
IEquatable Interface
Consider the following generic method :
Podcast for developers
Podcasts are a great way of utilizing your time while you are on the move i.e. driving car or riding in a bus/train. If you are unlucky like me, who has to spend couple of hours each day travelling between office & home and vice versa, then listening to podcasts while you are stuck in traffic can be of great help. Not only are you learning something new but it also helps you in keeping calm and makes the journey appear comparatively shorter.
Dependency Injection in PowerShell
In this blog post, I will explain how we can invoke dependency injection based managed code from PowerShell. Invoking regular managed code from PowerShell is quiet straight forward. Say for example, you are asked to create an instance of HttpClient class and call the GetStringAsync method on it, then it can be done with just following few lines of code.
2012
Laptop Inauguration
Today I received my new laptop which is an Intel Core i5-2450M @ 2.50GHz 4 GB RAM machine . The other laptop(office provided) which I have used for past two years for programming is an Intel Core2 Duo T6570 @ 2.10GHz machine. Reason why I am talking about the laptops that I own is because of my interest in writing multi-threaded/parallel code using the new TPL API provided in the .Net 4.0 framework. I have spent significant amount of time in past one year writing code using the Parallel API of .Net framework. But given the fact that the hardware that I was using for running those applications wasn’t that great, I was never much satisfied with the overall improvement in performance.