converting normal java code to hadoop code without mapreduce? -


I am currently working on the hoop. I want to change the map to my Java code. I want to code my code with hdfs i.e. my code is normal file system, i want to work with hdfs (hd file system) I want to work below code in Hood (HD File System) 

  import java.io.file; Import java.io.FileInputStream; Import java.io.FileOutputStream; Import java.io.IOException; Import java.util.zip.GZIPInputStream; Import java.util.zip.GZIPOutputStream; Public class GZIPExample {public static zero gzip () {int i = new file ("/ media / 0052ADF152ADEC1A / all partitions") List (). Length; System.out.println (I + "here"); While (i> gt; {} string file = "/ media / 0052ADF152ADEC1A / all partitions / files" + i + ".txt"; String gzipFile = "/ media / 0052ADF152ADEC1A / collapses / filegages" + i + ".gz"; String Newfile = "/ Media / 0052ADF152ADEC1A / All / ThEup Ebook / Test 1.txt"; CompressGzipFile (file, gzipFile); Decompress Jump File (gzipFile, newFile); I--; }} Private Static void decompressGzipFile (string gzipFile, String newFile) {try {fileInputStream fis = new FileInputStream (gzipFile); GJIPI inputstream GIS = new GJIPI inputstream (FIS); FileOutputStream fos = New FileOutputStream (newFile); Byte [] buffer = new byte [1024]; Int len; While ((len = gis.read (buffer)) = -1) {fos.write (buffer, 0, lane); } // close resources fos.close (); Gis.close (); } Hold (IOException e) {e.printStackTrace (); }} Private Static Zero compressGzipFile (string file, string gzipFile) {try {fileInputStream fis = new FileInputStream (file); FileOutputStream fos = New FileOutputStream (gzipFile); GZIPOutputStream gzipOS = new GZIPOutputStream (fos); Byte [] buffer = new byte [1024]; Int len; While ((len = fis.read (buffer))! = -1) {gzipOS.write (buffer, 0, lane); } // close processing gzipOS.close (); Fos.close (); Fis.close (); } Hold (IOException e) {e.printStackTrace (); }}}}}    
 
 
 I recommend you to read and understand that a MapReduce What the code looks like and what it needs to achieve:  
  
 Then you will understand that "conversion" is not in a very high level view, map code is modified in two steps. There is a way to do this: reduce the map and  
 from your example, it seems that you can use a large amount of files to be combined Tax / want to narrow parallel computing so you should try to write your MapReduce code as a two-stage process for joint / contraction to a file. Sorry, but I have never encountered compression algorithms.

Comments

Post a Comment

Popular posts from this blog

Java - Error: no suitable method found for add(int, java.lang.String) -

I'm in the middle of homework work and I'm stuck. At this point in my code, I think that I should have a GUI window that opens and allows me to type "inserted text number". Notice is not going anywhere but at this point, If I pass through the problem then it will be in a linked list. I am getting two of the same error for the lines. Add (index, element); And I can not seem to get past it, there is no suitable method for the error "add (int, java. string string)". Code is below, please advise. To clarify - this will not be a method error because it is a linked list. There should not be any way involved. import java.awt. *; Import java.awt.event. *; Import javax.swing. *; Import java.util. *; Import java.util.Scanner; Import java.util.LinkedList; Public class TopTenList JFrame {Private TopTenList tt; See the Private JTextArea list; Private JTextField cmdTextField; Private JTextField resultsTestfield; // This GUI window is the code for the public toptist...
Read more

java - JPA TypedQuery: Parameter value element did not match expected type -

I am using JPA 2.0 and getting the following code in the DAO layer: Public Zero Test () {string key = "status"; String [] Conditions = {"A", "B"}; & Lt; TestTable & gt; Results = Search (Keys, Conditions); } Public listing & lt; TestTable & gt; Search (string key, object value error) {string sql = "test ndf to testlet and jade nand." + Key + "in:" + key; TypedQuery & LT; TestTable & gt; Query = entityManager.createQuery (SQL, TestTable.class); Query.setParameter (key, Arrays.asList (valueArr)) / *** Error Line *** Return query.getResultList (); } In the above error line, it throws the following exception: java.lang.IllegalArgumentException: parameter value element [[Ljava.lang.String; @ cbe5bc] did not match the expected type [java.lang.String] Why is this expected type string while actually this string []? Please help! Note: This is derived from the normal routine and is a simplified code. I can no...
Read more

c++ - static template member variable has internal linkage but is not defined -

Yes, I know, there is a question with almost the same title, but it refers to a different situation. Error message) In my case, I have a .cpp file with a big named designation name (implementation details). There is a property class template with a static data member in that name space, which I need to access from outside the unknown namespace. I give it a little bit of meat: file.hpp namespace Bar {template & lt; Typename A & gt; Struct foo {static_assert (is_same & lt; a, float & gt; :: value} is_same & lt; a, double>: value, ""); Fixed zero set_static_var (a const & x); // ...}; } and file.cpp namespace {template & lt; Typename A & gt; Struct foo_traits {// supports the implementation of several static code bars: foo & lt; & Gt; Fixed A data; }; The template's & lt; & Gt; Float foo_traits & lt; Float & gt; :: datum; // No change if this is in the global namespace template & lt; & Gt; Doub...
Read more