Wondering what’s next for npm?Check out our public roadmap! »

    hdfs247

    1.0.0 • Public • Published

    NPM HDFS module

    Build Status

    This is node.js interface to communicating with web HDFS interface.

    How to use

    Installing

    Before we can use this module we need to install it first

    $ npm install --save hdfs247

    Using on Unsecure Cluster

    var Hdfs = require('hdfs'),
      localpath = '/tmp/filetest.txt',
      remotepath = '/user/apps/filetest.txt';
     
    var hdfs = new Hdfs({
      protocol: 'http',
      hostname: '192.168.1.225',
      port: 50070
    });
     
    hdfs.upload({
      'user.name': 'apps',
      overwrite: true,
      localpath: localpath,
      path: remotepath
    }, function(error, response, body) {
      console.log(error);    // Error will be null if upload process is succeed.
      console.log(response); // Raw response from node.js request.
      console.log(body);     // Body of request result.
    });

    Using on Secure Cluster (Kerberized/SPNego)

    var Hdfs = require('hdfs'),
      localpath = '/tmp/filetest.txt',
      remotepath = '/user/apps/filetest.txt';
     
    var hdfs = new Hdfs({
      protocol: 'http',
      hostname: '192.168.1.225',
      port: 50070
    });
     
    hdfs.upload({
      'user.name': 'apps',
      overwrite: true,
      localpath: localpath,
      spnego_token: result.token,
      principal: spnego_principal,
      keytab: keytab,
      realm: realm,
      krb_fqdn: krb_fqdn,
      path: remotepath
    }, function(error, response, body) {
      console.log(error);    // Error will be null if upload process is succeed.
      console.log(response); // Raw response from node.js request.
      console.log(body);     // Body of request result.
    });

    Functions

    cekpath(param)Object

    Check the parameter, return path if there is path in parameter, and return / if no path in parameter.

    prepvar(param)Object

    Prepare parameter, separate request option and request parameter, set default option, and explicitly set path.

    Hdfs(data)

    Hdfs class constructor

    cekpath(param) ⇒ Object

    Check the parameter, return path if there is path in parameter, and return / if no path in parameter.

    Kind: global function
    Returns: Object - Object with only path in it.

    Param Type Description
    param Object Parameter from user

    prepvar(param) ⇒ Object

    Prepare parameter, separate request option and request parameter, set default option, and explicitly set path.

    Kind: global function
    Returns: Object - Clean ready to use request parameter.

    Param Type Description
    param Object Mixed request parameter.

    Hdfs(data)

    Hdfs class constructor

    Kind: global function

    Param Type Description
    data Object Object data seed.

    hdfs.getfilestatus(param, callback)

    Get file status from HDFS Possible properties of param is :

    name presence type default description
    path optional String / Path status that will be taken

    Kind: instance method of Hdfs

    Param Type Description
    param Object Request parameter for get file status endpoint
    callback function Callback function to return the result

    hdfs.liststatus(param, callback)

    List status of file and directory under specified directory

    name presence type default description
    path optional String / Path status that will be taken

    Kind: instance method of Hdfs

    Param Type Description
    param Object Request parameter for get list of status file
    callback function Callback function to return the result

    hdfs.download(param, callback)

    Download file from HDFS with without buffering.

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Open_and_Read_a_File

    Param properties :

    name presence type default valid value description
    path optional String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    offset optional Long 0 >=0 The starting byte position
    length optional Long null >=0 or null The number of bytes to be processed. null means the entire file
    buffersize optional Int Specified in hadoop configuration >0 The size of the buffer used in transferring data.
    Param Type Description
    param Object Request parameter for download file from HDFS
    callback function Callback function to return the result

    hdfs.upload(param, callback)

    Upload file to HDFS without buffering.

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Create_and_Write_to_a_File

    Param properties :

    name presence type default valid value description
    path optional String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    overwrite optional Boolean false true or false If a file already exists, should it be overwritten?
    blocksize optional Long Specified in hadoop configuration >0 The block size of a file.
    replication optional Short Specified in hadoop configuration >0 The number of replications of a file.
    permission optional octal 755 0 - 1777 The permission of a file/directory.
    buffersize optional Int Specified in hadoop configuration >0 The size of the buffer used in transferring data.
    Param Type Description
    param Object Request parameter for upload file from HDFS
    callback function Callback function to return the result

    hdfs.open(param, callback)

    Open file from HDFS, this function will buffer content file in memory

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Open_and_Read_a_File

    Param properties :

    name presence type default valid value description
    path optional String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    offset optional Long 0 >=0 The starting byte position
    length optional Long null >=0 or null The number of bytes to be processed. null means the entire file
    buffersize optional Int Specified in hadoop configuration >0 The size of the buffer used in transferring data.
    Param Type Description
    param Object Request parameter for open file from HDFS
    callback function Callback function to return the result

    hdfs.gethomedirectory(param, callback)

    Get home directory of current accessing user.

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Get_Home_Directory

    Param Type Description
    param Object Request parameter for getting home directory.
    callback function Callback function to return the result

    hdfs.getcontentsummary(param, callback)

    Get summary of content

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Get_Content_Summary_of_a_Directory

    name presence type default valid value description
    path optional String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    Param Type Description
    param Object Request parameter for getting home directory.
    callback function Callback function to return the result

    hdfs.getfilechecksum(param, callback)

    Get checksum of file

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Get_File_Checksum

    name presence type default valid value description
    path optional String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    Param Type Description
    param Object Request parameter for get checksum of file.
    callback function Callback function to return the result

    hdfs.concat(param, callback)

    Concatenating file to another one. Warning : It's still experimental. We warn you.

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Concat_Files

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    sources Mandatory String A list of comma seperated absolute FileSystem paths without scheme and authority. A list of source paths.
    Param Type Description
    param Object Request parameter for concatenating file.
    callback function Callback function to return the result

    hdfs.append(param, callback)

    Append data to another file in HDFS

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Append_to_a_File

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    buffersize optional Int Specified in hadoop configuration >0 The size of the buffer used in transferring data.
    Param Type Description
    param Object Request parameter for upload file from HDFS
    callback function Callback function to return the result

    hdfs.create(param, callback)

    Create file in HDFS.

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Create_and_Write_to_a_File

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    overwrite optional Boolean false true or false If a file already exists, should it be overwritten?
    blocksize optional Long Specified in hadoop configuration >0 The block size of a file.
    replication optional Short Specified in hadoop configuration >0 The number of replications of a file.
    permission optional octal 755 0 - 1777 The permission of a file/directory.
    buffersize optional Int Specified in hadoop configuration >0 The size of the buffer used in transferring data.
    Param Type Description
    param Object Request parameter for upload file from HDFS
    callback function Callback function to return the result

    hdfs.rename(param, callback)

    Rename file or directory in HDFS

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Rename_a_FileDirectory

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    destination Mandatory String (an invalid path) An absolute FileSystem path without scheme and authority. The destination path.
    Param Type Description
    param Object Request parameter for rename file or directory.
    callback function Callback function to return the result

    hdfs.mkdirs(param, callback)

    Make directory in Hdfs

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Make_a_Directory

    Param properties :

    name presence type default valid value description
    path optional String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    permission optional octal 755 0 - 1777 The permission of a file/directory.
    Param Type Description
    param Object Request parameter for make directory in HDFS
    callback function Callback function to return the result

    hdfs.setpermission(param, callback)

    Set permission of file in HDFS

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Set_Permission

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    permission Mandatory octal 755 0 - 1777 The permission of a file/directory.
    Param Type Description
    param Object Request parameter for set permission of file and directory in HDFS
    callback function Callback function to return the result

    hdfs.setowner(param, callback)

    Set owner of file or directory

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Set_Owner

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    owner Optional String (means keeping it unchanged) Any valid username. The username who is the owner of a file/directory.
    group Optional String (means keeping it unchanged) Any valid group name. The name of a group.
    Param Type Description
    param Object Request parameter for set owner in HDFS
    callback function Callback function to return the result

    hdfs.setreplication(param, callback)

    Set replication factor of file.

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Set_Replication_Factor

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    replication Optional Short Specified in the configuration. >0 The number of replications of a file.
    Param Type Description
    param Object Request parameter for set replication file.
    callback function Callback function to return the result

    hdfs.settimes(param, callback)

    Set time and date of file in HDFS

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Set_Access_or_Modification_Time

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    modificationtime Optional Long -1 (means keeping it unchanged) -1 or a timestamp The modification time of a file/directory.
    accesstime Optional Long -1 (means keeping it unchanged) -1 or a timestamp The access time of a file/directory.
    Param Type Description
    param Object Request parameter for setting time and date of file.
    callback function Callback function to return the result

    hdfs.delete(param, callback)

    Delete file or directory in HDFS

    Kind: instance method of Hdfs
    See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Delete_a_FileDirectory

    Param properties :

    name presence type default valid value description
    path Mandatory String / An absolute FileSystem path without scheme and authority. Path status that will be taken
    recursive Optional Boolean false true or false Should the operation act on the content in the subdirectories?
    Param Type Description
    param Object Request parameter for deleting file and or directory.
    callback function Callback function to return the result

    Install

    npm i hdfs247

    DownloadsWeekly Downloads

    22

    Version

    1.0.0

    License

    MIT

    Unpacked Size

    53.6 kB

    Total Files

    7

    Last publish

    Collaborators

    • avatar