Universe

spacy-cleaner

Easily clean text with spaCy!

spacy-cleaner utilises spaCy Language models to replace, remove, and mutate spaCy tokens. Cleaning actions available are:

  • Remove/replace stopwords.
  • Remove/replace punctuation.
  • Remove/replace numbers.
  • Remove/replace emails.
  • Remove/replace URLs.
  • Perform lemmatisation.

See our docs for more information.

Example

import spacy import spacy_cleaner from spacy_cleaner.processing import removers, replacers, mutators model = spacy.load("en_core_web_sm") pipeline = spacy_cleaner.Pipeline( model, removers.remove_stopword_token, replacers.replace_punctuation_token, mutators.mutate_lemma_token, ) texts = ["Hello, my name is Cellan! I love to swim!"] pipeline.clean(texts) # ['hello _IS_PUNCT_ Cellan _IS_PUNCT_ love swim _IS_PUNCT_']

View more
Author info

Cellan Hall

GitHubCe11an/spacy-cleaner

Categories extension

Found a mistake or something isn't working?

If you've come across a universe project that isn't working or is incompatible with the reported spaCy version, let us know by opening a discussion thread.


Submit your project

If you have a project that you want the spaCy community to make use of, you can suggest it by submitting a pull request to the spaCy website repository. The Universe database is open-source and collected in a simple JSON file. For more details on the formats and available fields, see the documentation. Looking for inspiration your own spaCy plugin or extension? Check out the project idea section in Discussions.

Read the docsJSON source