13 packages found

M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

Keywords

Publisher

published 3.0.58 months ago
M
Q
P

Description

🤖 detect bots/crawlers/spiders via the user agent.

Keywords

Publisher

published 3.0.816 days ago
M
Q
P

Description

It uses the user-agents.org xml file for detecting bots.

Keywords

Publisher

published 1.0.105 years ago
M
Q
P

Description

detects bots/crawlers/spiders via the user agent.

Keywords

Publisher

published 2.3.02 years ago
M
Q
P

Description

Site mapper utilizing puppeteer to generate a xml sitemap

Keywords

Publisher

published 1.0.5a month ago
M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

Keywords

Publisher

published 2.1.172 months ago
M
Q
P

Description

isbot Bundle UMD

Keywords

Publisher

published 1.0.44 months ago
M
Q
P

Description

A jQuery plugin that helps you to hide your email on your page and prevent crawlers to get it!

Keywords

Publisher

published 0.1.05 years ago
M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

Keywords

Publisher

published 3.0.22 years ago
M
Q
P

Description

Parse robot directives within HTML meta and/or HTTP headers.

Keywords

Publisher

published 0.4.03 years ago
M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers. (Extended version by mastixmc)

Keywords

Publisher

published 3.2.0a year ago
M
Q
P

Description

Opensource Framework Crawler in Node.js

Keywords

Publisher

published 2.0.102 years ago
M
Q
P

Description

Crawler made simple

Keywords

Publisher

published 0.0.36 years ago