Sunday, March 31, 2013

Hadoop : How to have nested directories as input path ?

At times our input directories have sub-directories in them.
And when we add such directory as our input path we are bound to get exception like follows: Not a file
Say we have the following directory structure
Hadoop supports input paths to be a regular expression. I haven’t experimented with a lot of complex regex, but the simple placeholders ? and * does work.
Now, say, we only want to have the input to be from the directories named abcdefg, this can be achieved by having the input path to be something like follows:
To explain this:
In order to exclude sub-directory inputDirectoryNotNeededAsInput and have all the directories with names starting from inputSubDirectory, we have the following construct in the input path:
In order to exclude sub-directories named 1234 (denote that these directories have names with just 4 letters) and have all the directories with names having more than 4 letters eg : abcdefg, we have the following construct in the input path:


  1. This is a great inspiring tutorials.I am pretty much pleased with your good work.You put really very helpful information. Keep it up.
    Hadoop Training in hyderabad

    1. I have read your blog its very attractive and impressive. I like it your blog.

      JavaEE Training in Chennai JavaEE Training in Chennai

      Java Training in Chennai Core Java Training in Chennai Core Java Training in Chennai

      Java Online Training Java Online Training Core Java 8 Training in Chennai Java 8 Training in Chennai

    2. Hibernate Online Training Hibernate Online Training Hibernate Training in Chennai Hibernate Training in Chennai Java Online Training Java Online Training Hibernate Training Institutes in ChennaiHibernate Training Institutes in Chennai

  2. Thanks for your post; selenium is most trusted automation tool to validate web application and browser. This tool provides precise and complete information about a software application or environment. Selenium Training in Chennai | Selenium Course in Chennai | Best Selenium training institute in Chennai

  3. Latest technology have created a greater impact over testing web applications. This vital in identifying important issues that raises in web appplications. Thanks for sharing this information in here. Keep blogging article like this.

    Selenium training in chennai | Best selenium training institute in chennai | Selenium testing course in chennai

  4. This data is great and amazing. A debt of gratitude is in order for taking an ideal opportunity to talk about this, I feel upbeat about it and I adore adapting more about this theme. I utilize your manual for teach my understudies.
    Selenium Training institute in Chennai | Selenium Training in Chennai | Selenium Training

  5. Thanks for posting the blog. I am new to this site, but I felt comfortable while reading the post. Keep posting on topics like this.
    Oracle Fusion SCM Training

  6. I wish to show thanks to you just for bailing me out of this particular
    trouble.As a result of checking through the net and meeting
    techniques that were not productive, I thought my life was training in chennai

  7. Great information. I have got some important suggestions from it. Thank you for sharing.
    Best VMware Training Institute in Chennai | Best VMware Training Institute in Velachery

  8. I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.
    aws training in Chennai

  9. It is very useful information. It will help to improve my knowledge in Selenium. Thank you for sharing this awesome site.
    Selenium Training in Chennai | Selenium Training | Selenium Course in Chennai | Selenium Training Institute in Chennai

  10. Great blog! Thanks for giving such valuable information, this is unique one. Really admired.

    Selenium Training in Chennai

  11. Great post! This is very useful for me and gain more information, Thanks for sharing with us.

    Article submission sites

  12. very nice blogs!!! i have to learning for lot of information for this sites...Sharing for wonderful information. Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing

    Best AWS Training Academy in Kanchipuram

  13. Thanks for this greatful information. all this information is very important to all the users and can be used good at all this process
    Best Tally ERP 9.0 Training Academy in Kanchipuram


Any feedback, good or bad is most welcome.


Email *

Message *