Universe

textpipe

clean and extract metadata from text

textpipe is a Python package for converting raw text in to clean, readable text and extracting metadata from that text. Its functionalities include transforming raw text into readable text by removing HTML tags and extracting metadata such as the number of words and named entities from the text.

Example

from textpipe import doc, pipeline sample_text = 'Sample text! <!DOCTYPE>' document = doc.Doc(sample_text) print(document.clean) 'Sample text!' print(document.language) # 'en' print(document.nwords) # 2 pipe = pipeline.Pipeline(['CleanText', 'NWords']) print(pipe(sample_text)) # {'CleanText': 'Sample text!', 'NWords': 2}
Author info

Textpipe Contributors

GitHubtextpipe/textpipe

Categories standalone

Submit your project

If you have a project that you want the spaCy community to make use of, you can suggest it by submitting a pull request to the spaCy website repository. The Universe database is open-source and collected in a simple JSON file. For more details on the formats and available fields, see the documentation. Looking for inspiration your own spaCy plugin or extension? Check out the project idea label on the issue tracker.

Read the docsJSON source