A Simple Apache Spark Demo

Apache Spark is a data processing framework that can quickly perform processing tasks on very large data sets, and can also distribute data processing tasks across multiple computers, either on its own or in tandem with other distributed computing tools

About this example

In this post I am sharing a simple Apache Spark example project. The source code used for this example is available here: https://github.com/jobinesh/apache-spark-examples.git

Here is the quick overview of the modules that you may find in this project

  • spark-job-common :  All common classes that you need for building a Spark job are parked here. This approach may help you to avoid boilerplate code in your Spark job implementation
  • spark-job-impl : A classic word count Spark  example is available here.   This class may help you to understand the structuring of the source and usage of common classes from spark-job-common module
  • spark-job-launcher : The SparkLauncher helps you to start Spark applications programmatically.

The Spark application that you may find in the spark-job-impl module reads the text from src/main/resources/demo.txt file
and generates an output file with total count for each word. 
The output directory location is configured in src/main/resources/application.conf

How to run this example?

The detailed steps are available here: 

Enjoy !
   

Comments

  1. I will truly value the essayist's decision for picking this magnificent article fitting to my matter.Here is profound depiction about the article matter which helped me more.
    data science courses in malaysia

    ReplyDelete
  2. Great blog!! I hope you'll post a Data Science blog here.
    data science course in delhi

    ReplyDelete
  3. Hello there to everyone, here everybody is sharing such information, so it's fussy to see this webpage, and I used to visit this blog day by day
    hrdf training course

    ReplyDelete
  4. You re in motivation behind fact an on-target site administrator. The site stacking speed is amazing. It kind of feels that you're doing a specific trick. What's more, The substance is a masterpiece. you have done a marvelous development concerning this issue!
    360DigiTMG

    ReplyDelete
  5. This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information...
    data science training

    ReplyDelete
  6. While most arrangements at second hand shops are very acceptable contrasted with retail, periodically somebody doing the estimating will put a label that is substantially a lot for a thing.besimple.com/

    ReplyDelete
  7. Thank you for your post, I look for such article along time, today i find it finally. this post give me lots of advise it is very useful for me !data science training in Hyderabad

    ReplyDelete

  8. keep up the good work. this is an Ossam post. This is to helpful, i have read here all post. i am impressed. thank you. this is our site please visit to know more information
    data science courses

    ReplyDelete
  9. It is very helpful content. Thanks to share.
    hosting server

    ReplyDelete
  10. Excellent post. I was always checking this blog, and I’m impressed! Extremely useful info specially the last part, I care for such information a lot. I was exploring this particular info for a long time. Thanks to this blog my exploration has ended.
    If you want Digital Marketing Serives :-
    Digital marketing Service in Delhi
    SMM Services
    PPC Services in Delhi
    Website Design & Development Packages
    SEO Services PackagesLocal SEO services
    E-mail marketing services
    YouTube plans

    ReplyDelete
  11. Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts.
    Digital Marketing Training Institutes in Hyderabad

    ReplyDelete
  12. I’ve been searching for some decent stuff on the subject and haven't had any luck up until this point, You just got a new biggest fan!..
    business analytics course

    ReplyDelete
  13. Incredibly conventional blog and articles. I am realy very happy to visit your blog. Directly I am found which I truly need. Thankful to you and keeping it together for your new post.
    data science certification malaysia

    ReplyDelete
  14. And if so be the case then, demand for people with strong programming skills will definitely rise with more number of people required to develop and support the applications. data science course in india

    ReplyDelete
  15. Nice Blog !
    Need quick help in QuickBooks? Don’t get annoyed!! Just reach out to our QuickBooks Phone Number 1-855-652-7978, and acquire excellent service benefits. We have a technical team of QB experts who can troubleshoot all sorts of complicated error codes without taking too much time

    ReplyDelete
  16. Great blog!! I hope you'll post a Data Science blog here. hair transplant turkey success rate

    ReplyDelete
  17. This article content is really unique and amazing. This article really helpful and explained very well's i am really thankful to you for sharing keep it up..

    Digital Marketing Service in Kanpur

    ReplyDelete
  18. Really glad to say, your post is very interesting to read. I never stop myself to say anything about it. You are doing a great job. Thank you for sharing this blog here
    Singapore Online Job Vacancy 2021

    ReplyDelete
  19. Incredibly conventional blog and articles. I am realy very happy to visit your blog. Directly I am found which I truly need. Thankful to you and keeping it together for your new post.
    data science courses in gurgaon

    ReplyDelete
  20. Amazing Article. I would also suggest for Data Science course with Real time experience, visit: https://socialprachar.com/data-science-training-in-bengaluru/

    ReplyDelete
  21. This is excellent information about the job which is shared by you. This information is meaningful and magnificent for us to increase our knowledge about it. Keep sharing this kind of information. Thank you. Jobs in Luton

    ReplyDelete
  22. Very awesome!!! When I searched for this I found this website at the top of all blogs in search engines.

    business analytics course

    ReplyDelete
  23. Thanks for sharing A Simple Apache Spark Demo. That is very useful to me as Immigration Consultants Singapore

    ReplyDelete


  24. Really impressed! Everything is very open and very clear clarification of issues. It contains true facts. Your website is very valuable. Thanks for sharing.

    Data Science Training in Hyderabad

    ReplyDelete
  25. It's really nice and meaningful. it's a really cool blog. Linking is a very useful thing.you have really helped lots of people who visit blogs and provide them useful information.

    Data Science Training

    ReplyDelete
  26. This article is exciting. I am pleased to read it. If you wonder what every type of papers should look like, visit https://primeessays.com/samples/world-literature/the-necklace-by-guy-de-maupassant.html There are hundreds of free samples pf essays.

    ReplyDelete
  27. I must appreciate you for providing such valuable content for us. To make our dreams a reality, everyone has to put up a hard struggle in their life. Here I am making my small effort to arrange up the list of highest paying Jobs in India. We provide one of the best courses for you.
    1) Data Science course in Delhi
    2) Artificial Intelligence Course in Delhi
    3) Machine Learning course in Delhi
    4) Business Analytics courses in Delhi
    5) Digital marketing course in Delhi
    6) Full Stack Software Developer course in Delhi.

    ReplyDelete

  28. Very awesome!!! When I searched for this I found this website at the top of all blogs in search engines.

    Data Science Training

    ReplyDelete
  29. Your blog is very valuable which you have shared here about JOBS I appreciate your efforts which you have put into this article and also it is a gainful article for us. Thank you for sharing this article here. Careers working with developmental disabilities

    ReplyDelete
  30. I read your article and after I read got beneficial information about the jobs. Your article contains excellent information and knowledge. It is beneficial for all. .Jobs in BC CanadaKeep sharing.

    ReplyDelete
  31. The information you have posted is very useful. The sites you have referred to were good. Thanks for sharing.
    data scientist training and placement in hyderabad

    ReplyDelete
  32. I have expressed a few of the articles on your website now, and I really like your style of blogging. I added it to my favorite’s blog site list and will be checking back soon…

    best data science institute in hyderabad

    ReplyDelete
  33. Download Cricket Mazza 11 for Android Live Line & Fastest IPL Score. Get Live Cricket Scores, Scorecard, Commentary, Match Info and Schedules of All International & Domestic Matches, Series wise Stats, Records, Analysis and Facts, Trending News and Tweets, Recent ICC Player and Team Rankings. Also Download best cricket game apps for ios, including Beach Cricket, Cricket T20 Fever 3D, Download Real Cricket mazza 11 and other top answers suggested and ranked.

    ReplyDelete
  34. Your text was very informative for me but I'd like to add that in terms of the set of components and functionality, Spark can be compared with another Big Data distributed stream processing tool - Apache Flink. By the way if you need to pass paper works I would like to recommend very good service where you can buy pre written essays on any topic and subject!

    ReplyDelete
  35. I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it.
    business analytics course


    ReplyDelete
  36. I read that Post and got it fine and informative. Please share more like that...
    Best Data Science courses in Hyderabad

    ReplyDelete
  37. This comment has been removed by the author.

    ReplyDelete
  38. Hi! Excellent blog which describes all the A Simple Apache Spark Demo of the blog in detail. If you want to know about server hosting or interested in best USA VPS Hosting you can ask us for more details and services. Get best and fast hosting service with us. Don't think get fastest hosting services with us. Buy one of the fastest and reliable hosting services

    ReplyDelete


  39. Nice post. I used to be checking constantly this blog and I am impressed! Extremely useful info particularly the ultimate section 🙂 I take care of such information a lot. I was seeking this certain information for a long time. Thank you and best of luck.
    essay on the person i admire the most

    ReplyDelete

Post a Comment