367 packages found

    Description

    Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously

    Keywords

    Publisher

    published 1.3.05 months ago
    M
    Q
    P

    Description

    Crawler (spider) of site web pages by domain name

    Keywords

    Publisher

    published 1.2.02 months ago
    M
    Q
    P

    Description

    xvideos.com api implementation and update.

    Keywords

    Publisher

    published 1.1.214 months ago
    M
    Q
    P

    Description

    har download 从 chrome 生成的 har 文件中下载整个网站所有资源

    Keywords

    Publisher

    published 2.1.06 months ago
    M
    Q
    P

    Description

    Parse and stringify cookies for web spider.

    Keywords

    Publisher

    published 1.0.3a month ago
    M
    Q
    P

    Description

    极简网络蜘蛛爬虫,适用任何网站,只需设置一条规则,就可以把你想要网站上的内容整理出来,非常方便,简单!

    Keywords

    Publisher

    published 5.0.105 months ago
    M
    Q
    P

    Description

    This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

    Keywords

    Publisher

    published 3.1.3a month ago
    M
    Q
    P

    Description

    Get a list of local URL links from a root URL.

    Keywords

    Publisher

    published 2.0.0a month ago
    M
    Q
    P

    Description

    DEX8 SDK is a library which helps developers to create DEX8 crawler, scraper tasks or automated serverless scripts.

    Keywords

    Publisher

    published 2.9.52 months ago
    M
    Q
    P

    Description

    A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS

    Keywords

    Publisher

    published 2.0.02 years ago
    M
    Q
    P

    Description

    And ExpressJS middleware for detecting search engine crawlers/spiders and handling crawler-specific logic such as additional logging for requests from spiders

    Keywords

    Publisher

    published 1.1.9a year ago
    M
    Q
    P

    Description

    爬虫工具,根据传入的配置和规则爬取数据

    Keywords

    Publisher

    published 1.1.4-0a year ago
    M
    Q
    P

    Description

    a crawler cli of comics

    Keywords

    Publisher

    published 1.3.35 months ago
    M
    Q
    P

    Description

    node.js web crawler

    Keywords

    Publisher

    published 2.0.12 years ago
    M
    Q
    P

    Description

    极简网络蜘蛛爬虫,适用任何网站,只需设置一条规则,就可以把你想要网站上的内容整理出来,非常方便,简单!

    Keywords

    Publisher

    published 5.0.102 years ago
    M
    Q
    P

    Description

    Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

    Keywords

    Publisher

    published 1.1.9a year ago
    M
    Q
    P

    Description

    Simple WAF to integrate with Node.js web systems

    Keywords

    Publisher

    published 1.0.3a year ago
    M
    Q
    P

    Description

    web spider, support puppeteer, cheerio and so on, include task-queue and dispatcher

    Keywords

    Publisher

    published 2.2.4-preview.16073501019664 months ago
    M
    Q
    P

    Description

    xvideos.com api implementation.

    Keywords

    Publisher

    published 1.1.324 days ago
    M
    Q
    P

    Description

    a weibo spider

    Keywords

    Publisher

    published 0.1.010 months ago
    M
    Q
    P