Have ideas to improve npm?Join in the discussion! »

@drorgl/sitemapperTypeScript icon, indicating that this package has built-in type declarations

2.1.17 • Public • Published

Sitemap-parser

Parse through a sitemaps xml to get all the urls for your crawler.

NOTICE

This is a fork of sitemapper by Sean Thomas Burke, converted to typescript and extended to return the full sitemap.xml schema.

Installation

npm install @drorgl/sitemapper --save

Simple Example

import Sitemapper from "@drorgl/sitemapper";
 
let sitemap = new Sitemapper();
 
const result = await sitemap.crawlSite(url);
 

Custom Request Limits

import Sitemapper from "@drorgl/sitemapper";
 
let sitemapper = new Sitemapper({
    requester: new SuperAgentRequester({
        timeout: 15000,
        maximum_parallelism: 5,
        parallelism_delay: 100,
        gracefulFailure: true
    })
});
 
const result = await sitemap.crawlSite(url);

Test

npm run test

Coverage

npm run coverage
=============================== Coverage summary ===============================
Statements   : 93.42% ( 71/76 )
Branches     : 76.92% ( 30/39 )
Functions    : 96.43% ( 27/28 )
Lines        : 92.96% ( 66/71 )
================================================================================

Documentation

npm run doc

Install

npm i @drorgl/sitemapper

DownloadsWeekly Downloads

1

Version

2.1.17

License

MIT

Unpacked Size

34.4 kB

Total Files

39

Last publish

Collaborators

  • avatar