11 packages found

M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

Keywords

Publisher

published 3.0.53 months ago
M
Q
P

Description

detects bots/crawlers/spiders via the user agent.

Keywords

Publisher

published 2.5.72 months ago
M
Q
P

Description

It uses the user-agents.org xml file for detecting bots.

Keywords

Publisher

published 1.0.105 years ago
M
Q
P

Description

detects bots/crawlers/spiders via the user agent.

Keywords

Publisher

published 2.3.02 years ago
M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers. (Extended version by mastixmc)

Keywords

Publisher

published 3.2.0a year ago
M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

Keywords

Publisher

published 2.1.149 months ago
M
Q
P

Description

A jQuery plugin that helps you to hide your email on your page and prevent crawlers to get it!

Keywords

Publisher

published 0.1.05 years ago
M
Q
P

Description

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

Keywords

Publisher

published 3.0.22 years ago
M
Q
P

Description

Parse robot directives within HTML meta and/or HTTP headers.

Keywords

Publisher

published 0.4.03 years ago
M
Q
P

Description

Opensource Framework Crawler in Node.js

Keywords

Publisher

published 2.0.10a year ago
M
Q
P

Description

Crawler made simple

Keywords

Publisher

published 0.0.35 years ago