Wednesday, June 11, 2014

What is the need to use job.setJarByClass in the driver section of Hadoop MapReduce solution?

Have you ever thought of this?

We do provide the jar to be executed by hadoop while executing hadoop command, don't we?

$ hadoop jar /some-jar.jar

So, why do we need to have the following line of code in our Driver section while declaring the Job object properties:
job.setJarByClass(SomeClass.class);

Answer to that is very simple. Here you help Hadoop to find out that which jar it should send to nodes to perform Map and Reduce tasks. Your some-jar.jar might have various other jars in it's classpath, also your driver code might be in a separate jar than that of your Mapper and Reducer classes.

Hence, using this setJarByClass method we tell Hadoop to find out the relevant jar by finding out that the class specified as it's parameter to be present as part of that jar. So usually you should provide either MapperImplementation.class or your Reducer implementation or any other class which is present in the same jar as that pf Mapper and Reducer. Also make sure that both Mapper and Reducer are part of the same jar.

63 comments:

  1. What if I don't specify it. I mean if I comment it off?

    ReplyDelete
  2. Great post. I really learned some new things in here. So thanks.

    catering services in chennai

    ReplyDelete
    Replies
    1. I have read your blog its very attractive and impressive. I like it your blog.

      Java Training in Chennai Core Java Training in Chennai Core Java Training in Chennai

      Java Online Training Java Online Training Core Java 8 Training in Chennai Core java 8 online training JavaEE Training in Chennai Java EE Training in Chennai

      Delete
    2. Hibernate Online Training Hibernate Online Training Hibernate Training in Chennai Hibernate Training in Chennai Java Online Training Java Online Training

      Hibernate Training Institutes in ChennaiHibernate Training Institutes in Chennai Java Online Training Java Online Training Java Online Training Java Online Training

      Delete
    3. Java Training Institutes Java Training Institutes Java EE Training in Chennai Java EE Training in Chennai Java Spring Hibernate Training Institutes in Chennai J2EE Training Institutes in Chennai J2EE Training Institutes in Chennai Core Java Training Institutes in Chennai Core Java Training Institutes in Chennai

      Delete
  3. IVR is a system that reply to human when the admin in not Available.... Human Sleep but IVR don't sleep.......

    IVR Application

    ReplyDelete
  4. My Arcus offer java training with 100% placement. Our java training course that includes fundamentals and advance java training program with high priority jobs. java j2ee training with placement having more exposure in most of the industry nowadays in depth manner of java .

    java training in chennai

    ReplyDelete
  5. Hi this is very informative and useful blog for all kinda peoples.

    PHP Training in Chennai

    ReplyDelete
  6. This is truly a standout amongst the most gainful online journals I've ever skimmed on this subject. I am exceptionally happy to peruse such an incredible blog and thank you for offering this great information to us.
    Spring Training in Chennai | Spring Hibernate Training | Core Spring Training

    ReplyDelete
  7. Nice post, Thanks for sharing this useful information with us.
    IT Job Training in Chennai

    ReplyDelete
  8. This post was quite awesome and interesting to read. Congrats for your work. Thanks a lot for providing this with us. Keep on updating this with us regularly.

    ReplyDelete
  9. This post was quite awesome and interesting to read. Congrats for your work. Thanks a lot for providing this with us. Keep on updating this with us regularly.

    ReplyDelete
  10. ERPTREE is a leading oracle fuison HCM Training Institute. we offer this course through online we have great experience in succeeding students through online courses. we can calculate our performance through their honest comments in our sites in supporting our services. we have referral program so candidates can earn money through referral. you can share your live experience with other can generate you some money.



    Oracle Fusion Cloud HCM Training

    ReplyDelete
  11. Nice and good informative article, thanks for sharing your valuable ideas, keep rocks.
    Software Testing Training in chennai | Software Testing Training in velachery

    ReplyDelete
  12. As an engineer by training, I had concentrated on effectiveness of process. In summary the job seeking system gets the job seeker ahead of the competition and closer to the hiring manager, effectively what a good recruiter does to win recruitment business. guarantor

    ReplyDelete
  13. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.

    Java Training in Bangalore

    ReplyDelete
  14. Informative post about hadoop, i am looking forward for realtime hadoop online training institute.

    ReplyDelete
  15. very Nice and good informative article, thanks for sharing your valuable ideas, keep rocks.
    Android Training Institute in Chennai | IOS Training in Chennai | Online Android Training in Velachery

    ReplyDelete
  16. Can’t get over the fact that words can be used this beautifully.
    Resumeyard

    ReplyDelete

  17. I’m glad you enjoyed it. Those are great habits! Thank you for sharing.
    lennyfacetext.com

    ReplyDelete
  18. Those guidelines additionally worked to become a good way to
    recognize that other people online have the identical fervor like mine
    to grasp great deal more around this condition.


    java Training in Bangalore


    big data Training in Bangalore

    ReplyDelete
  19. Thank you a lot for providing individuals with a very
    spectacular possibility to read critical reviews from this site.


    java training in chennai

    ReplyDelete
  20. I appreciated your work very thanks iti govt jobs

    ReplyDelete
  21. This is my first time visit to your blog and I am very interested in the articles that you serve. Provide enough knowledge for me. Thank you for sharing useful and don't forget, keep sharing useful info: Visit Office Rental Singapore

    ReplyDelete
  22. Great post will get very useful information who are looking towards this site.
    Best Online Software Training Institute | Big Data Analytics Training

    ReplyDelete
  23. Ciitnoida provides Core and java training institute in

    noida
    . We have a team of experienced Java professionals who help our students learn Java with the help of Live Base Projects. The object-

    oriented, java training in noida , class-based build

    of Java has made it one of most popular programming languages and the demand of professionals with certification in Advance Java training is at an

    all-time high not just in India but foreign countries too.

    By helping our students understand the fundamentals and Advance concepts of Java, we prepare them for a successful programming career. With over 13

    years of sound experience, we have successfully trained hundreds of students in Noida and have been able to turn ourselves into an institute for best

    Java training in Noida.

    java training institute in noida
    java training in noida

    ReplyDelete
  24. In our society majority of the children gets education for degree purpose and so that they make a good career but few people in this world thinks that the triune purpose of education is to make minds, not careers and
    Details cleck here..... NGO Jobs

    ReplyDelete
  25. Good one. Thanks. I am planning to do PGP in big data analytics from INSOFE as my frends recommended it as one of the best big data analytics training institute in Hyderabad. Any suggestions?

    ReplyDelete
  26. I got good information by reading this article.I learned Development and Operations.Thankyou for this articleDevOps Training In Chennai | DevOps Training Institute In Chennai

    ReplyDelete
  27. Thanks for sharing the information, Salesforce experts a lot of openings in multi-level companies, for more information n
    Salesforce Training
    Salesforce certification Training program
    Salesforce Online Training in Bangalore

    ReplyDelete
  28. Thanks for your article .its very useful for everyone. Hadoop training in chennai

    ReplyDelete
  29. myTectra Amazon Web Services (AWS) certification training helps you to gain real time hands on experience on AWS. myTectra offers AWS training in Bangalore using classroom and AWS Online Training globally. AWS Training at myTectra delivered by the experienced professional who has atleast 4 years of relavent AWS experince and overall 8-15 years of IT experience. myTectra Offers AWS Training since 2013 and retained the positions of Top AWS Training Company in Bangalore and India.

    ReplyDelete
  30. IOT Training in Bangalore - Live Online & Classroom
    IOT Training course observes iot as the platform for networking of different devices on the internet and their inter related communication. Reading data through the sensors and processing it with applications sitting in the cloud and thereafter passing the processed data to generate different kind of output is the motive of the complete curricula. Students are made to understand the type of input devices and communications among the devices in a wireless media.

    ReplyDelete
  31. mytectra placement Portal is a Web based portal brings Potentials Employers and myTectra Candidates on a common platform for placement assistance.

    ReplyDelete
  32. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command.

    aws training in chennai

    hadoop training in chennai

    ReplyDelete
  33. Nice tutorial. Thanks for sharing the valuable information. it’s really helpful. Who want to learn this blog most helpful. Keep sharing on updated tutorials…
    Selenium Training in Chennai | Selenium Training in Bangalore | Selenium Training in Pune | Selenium online Training

    ReplyDelete
  34. Thank you for sharing your article. Great efforts put it to find the list of articles which is very useful to know, Definitely will share the same to other forums.
    data science course in chennai quora | data science course fees in chennai | data science course in chennai velachery | data science course in chennai omr

    ReplyDelete
  35. The blog refers about aws and thanks for this type of post.
    AWS Training In Chennai

    ReplyDelete
  36. Very decent post and very informative for me. Thanks for sharing. Big Data Training in Pune

    ReplyDelete
  37. Thank you for sharing such great information with us. I really appreciate everything that you’ve done here and am glad to know that you really care about the world that we live in.
    Web Designing Course in chennai
    PHP Training in Chennai
    website designing training
    web designing training
    PHP Training Center in Chennai
    PHP Institutes in Chennai

    ReplyDelete
  38. Interesting Post. Looking for this information for a while. Thanks for Posting.

    superbfacts
    Technology

    ReplyDelete
  39. such an effective blog you are posted.this blog is full of innovative ideas and i really like your information's. i expect more ideas from your site please add more details in future.
    vmware training near me
    vmware training in chennai
    best institute for vmware in bangalore
    vmware training near me

    ReplyDelete
  40. This is a Really Good Article. It is Amazing. Thanks for sharing this article.
    Check more Government Jobs Circular

    ReplyDelete
  41. Thank you shares this article. I found this article more information. new government job circular

    ReplyDelete

Any feedback, good or bad is most welcome.

Name

Email *

Message *