Apache Spark is a data processing framework that can quickly perform processing tasks on very large data sets, and can also distribute data processing tasks across multiple computers, either on its own or in tandem with other distributed computing tools About this example In this post I am sharing a simple Apache Spark example project. The source code used for this example is available here: https://github.com/jobinesh/apache-spark-examples.git Here is the quick overview of the modules that you may find in this project spark-job-common : All common classes that you need for building a Spark job are parked here. This approach may help you to avoid boilerplate code in your Spark job implementation spark-job-impl : A classic word count Spark example is available here. This class may help you to understand the structuring of the source and usage of common classes from spark-job-common module spark-job-launcher : The SparkLauncher helps you to start Spark applications programmatical
- Other Apps
While building applications that comprises of multiple microservices, it is essential to have a mechanism in place to collect and analyze the details of API calls, timing data needed to troubleshoot latency problems and logging error generated from API calls. Jaeger is one such solution and used for monitoring and troubleshooting application built following microservice based architecture with following capabilities: Distributed context propagation Distributed transaction monitoring Root cause analysis Service dependency analysis Performance / latency optimization Jaeger is hosted by the Cloud Native Computing Foundation (CNCF) as the 7th top-level project (graduated in October 2019). Eclipse MicroProfile, OpenTracing and Jaeger As there are multiple tracing solutions out there similar to Jaeger(such as Zipkin), it is really good to avoid vendor lock-in by having some standardization around APIs that work with different providers.The OpenTracing addresses this part of the problem, i
- Other Apps
Redis is an open source, in-memory data structure store, used as a database, cache and message broker. In this short post I am sharing a simple application that show cases classic integration of Redis message broker(pub/sub) APIs with a Helidon MicroProfile application for message publish-subscribe use case. You can checkout the source form here: https://github.com/jobinesh/cloud-native-applications/blob/master/helidon-example-mp-redis/README.md What is the usecase exercised in this example ? It is simple :) We use the a simple greeting REST API to exercise the Redis Publish Subscribe feature . When a client updates the greeting message, the Greeting resource implementation will publish the new greeting message to Redis channel topic for use by interested parties(consumers) Who does what? Here is quick summary of the classes that you find in the source: Lettuce : This example use Lettuce client library to connect to Redis server. If you are new to Lettuce, take a look a
The views expressed on this blog are my own and do not necessarily reflect the views of my employer.