100 packages found

M
Q
P

Description

Tiny JavaScript tokenizer.

Keywords

Publisher

published 6.0.02 months ago
M
Q
P

Description

transform stream to tokenize html

Keywords

Publisher

published 2.0.12 months ago
M
Q
P

Description

A fast tokenizer/lexer core that is well tested.

Keywords

Publisher

published 0.2.16 months ago
M
Q
P

Description

Tokenize JavaScript

Keywords

Publisher

published 0.2.16 months ago
M
Q
P

Description

Tokenize Excel formulas

Keywords

Publisher

published 2.3.1a month ago
M
Q
P

Description

A simple, Twitter-aware tokenizer.

Keywords

Publisher

published 2.0.023 days ago
M
Q
P

Description

personal tokenize util

Keywords

Publisher

published 2.0.02 months ago
M
Q
P

Description

lexing json

Keywords

Publisher

published 1.1.14 years ago
M
Q
P

Description

Lexical analyzer built in Javascript

Keywords

Publisher

published 1.0.42 years ago
M
Q
P

Description

Transform stream that tokenizes CSS

Keywords

Publisher

published 1.0.15 years ago
M
Q
P

Description

String ngram splitter.

Keywords

Publisher

published 0.2.110 months ago
M
Q
P

Description

tokenize c/c++ source code

Keywords

Publisher

published 1.0.04 years ago
M
Q
P

Description

NLP Functions for amplifying negations, managing elisions, creating ngrams, stems, phonetic codes to tokens and more.

Keywords

Publisher

published 2.0.5a year ago
M
Q
P

Description

Simple synchronous string tokenizer using Regex

Keywords

Publisher

published 2.0.03 years ago
M
Q
P

Description

Tokenizes utterances that contain a mix of English and Chinese words.

Keywords

Publisher

published 1.1.02 years ago
M
Q
P

Description

Get search tokens from an INSA/NASA station name.

Keywords

Publisher

published 1.1.03 months ago
M
Q
P

Description

Binary wrapper for Sedlex_ppx - A lexer generator for OCaml, similar to ocamllex, but supporting Unicode.

Keywords

Publisher

published 20.0.06 months ago
M
Q
P

Description

A streaming JSON tokenizer

Keywords

Publisher

published 1.1.03 years ago
M
Q
P
M
Q
P

Description

A simple iterative lexer written in TypeScript

Keywords

Publisher

published 0.7.4a year ago