Connecting the {Dots}

Apache Spark is a data processing framework that can quickly perform processing tasks on very large data sets, and can also distribute data processing tasks across multiple computers, either on its own or in tandem with other distributed computing tools About this example In this post I am sharing a simple Apache Spark example project. The source code used for this example is available here:  https://github.com/jobinesh/apache-spark-examples.git Here is the quick overview of the modules that you may find in this project spark-job-common :  All common classes that you need for building a Spark job are parked here. This approach may help you to avoid boilerplate code in your Spark job implementation spark-job-impl  : A classic word count Spark  example is available here.   This class may help you to understand the structuring of the source and usage of common classes from spark-job-common module spark-job-launcher : The SparkLauncher helps you to start Spar...

While building applications that comprises of multiple microservices, it is essential to have a mechanism in place to collect and analyze the details of API calls, timing data needed to troubleshoot latency problems and logging error generated from API calls. Jaeger is one such solution and used for monitoring and troubleshooting application built following microservice based architecture with following capabilities: Distributed context propagation Distributed transaction monitoring Root cause analysis Service dependency analysis Performance / latency optimization Jaeger is hosted by the Cloud Native Computing Foundation (CNCF) as the 7th top-level project (graduated in October 2019). Eclipse MicroProfile, OpenTracing and Jaeger As there are multiple tracing solutions out there similar to Jaeger(such as Zipkin), it is really good to avoid vendor lock-in by having some standardization around APIs that work with different providers.The  OpenTracing  addresses this part of the p...

Redis is an open source, in-memory data structure store, used as a database, cache and message broker. In this short post I am sharing a simple application that show cases classic integration of Redis message broker(pub/sub) APIs with a Helidon MicroProfile application for message publish-subscribe use case. You can checkout the source form here:  https://github.com/jobinesh/cloud-native-applications/blob/master/helidon-example-mp-redis/README.md What is the usecase exercised in this example ? It is simple :) We use the a simple greeting REST API  to exercise the Redis Publish Subscribe feature . When a client updates the greeting message, the Greeting resource implementation will publish the new greeting message to Redis channel topic for use by interested parties(consumers) Who does what? Here is quick summary of the classes that you find in the source: Lettuce : This example use  Lettuce  client library to connect to Redis server. If you are new to Let...

This is in continuation of the my previous posts on gRPC CRUD Example with Java. In case you have not read them yet, here are the links - Pause here, take a look at those posts and then resume ;) https://www.jobinesh.com/2020/01/a-simple-grpc-example.html https://www.jobinesh.com/2020/01/a-simple-grpc-crud-example-with-java.html This post shares a simple and complete gRPC CRUD example that runs as Helidon SE microservice.  The gRPC service generated using stdndrad approach as we discussed in previous posts. However the this example runs the gRPC APIs on Helidon SE service ;) If you are interested to see how the Helidon SE embraces gRPC service implementation that we have, see this class:  HelidonSEServer.java  in the example project. As a bonus we can leverage the built-in Helidon SE offerings for monitoring and tracing the gRPC APIs. For instance the following URL gives you health check info for our example: http://localhost:8080/health You can find t...

This is in continuation of the my previous post on gRPC with Java . This post shares a simple and complete gRPC CRUD example with Java. You can find the complete source here:  https://github.com/jobinesh/cloud-native-applications/tree/master/grpc-hr-example The following APIs are exposed by the gRPC server used in this example: rpc createDepartment (Department) returns (Department) {}; rpc updateDepartment (Department) returns (Department) {}; rpc findDepartmentsByFilter(DepartmentFilter) returns (DepartmentList) {}; rpc deleteDepartment(google.protobuf.Int64Value) returns (google.protobuf.Empty) {}; rpc findDepartmentById (google.protobuf.Int64Value) returns (Department) {}; rpc findAllDepartments(google.protobuf.Empty) returns (stream Department) {}; rpc updateDepartmentsInBatch(stream Department) returns (stream Department){}; The README.md   has the detailed steps for running the example. Enjoy !

Sometime back I blogged about core projects under  the  Cloud Native Computing Foundation:  https://www.jobinesh.com/2020/01/a-glance-at-cloud-native-computing.html . It is time for us to revisit some of the core projects in detail to learn how to fit them in a real life cloud native application. In this post I will briefly touch one of the promising projects incubating under Cloud Native Computing Foundation:  gRPC What is gRPC ? The gRPC is  remote procedure call (RPC) framework and toolset developed at Google. This is built on top of HTTP/2, leveraging the the benefits of a platform independence of the HTTP. It essentially means that this protocol works with various programming languages :) Here is the definition from  wikipedia : gRPC (gRPC Remote Procedure Calls) is an open source remote procedure call (RPC) system initially developed at Google in 2015. It uses HTTP/2 for transport, Protocol Buffers as the interface description language, and pr...

Here is the quick list of commands that one need to use in day to day life. As I keep forgetting them, thought of listing down in one place.  Hopefully this list may save my time ;) Configuring your identity  $ git config --global user.name "FirstName LastName" $ git config --global user.email firstname.lastname@somedomain.com Clone repository $ git clone Create new branch $ git  checkout -b feature/branch-name Staging and committing changes $ git  add . $ git  commit -m "comments" $ git push -u origin feature/branch-name Viewing git log $ git log --after="2020-01-01 00:00" $ git log --all --pretty=format:"%h %cd %s (%an)" --since=7.days $ git log --all --pretty=format:"%h %cd %s (%an)" --since='7 days ago' Rebasing a feature branch $ git checkout master $ git pull $ git checkout feature/branch-name $ git rebase master $ git add $ git rebase --continue $ git rebase master $ git push origin fe...