Pipeline

EntityRuler

class
String name:entity_rulerTrainable:
Pipeline component for rule-based named entity recognition

The entity ruler lets you add spans to the Doc.ents using token-based rules or exact phrase matches. It can be combined with the statistical EntityRecognizer to boost accuracy, or used on its own to implement a purely rule-based entity recognition system. For usage examples, see the docs on rule-based entity recognition.

Assigned Attributes

This component assigns predictions basically the same way as the EntityRecognizer.

Predictions can be accessed under Doc.ents as a tuple. Each label will also be reflected in each underlying token, where it is saved in the Token.ent_type and Token.ent_iob fields. Note that by definition each token can only have one label.

When setting Doc.ents to create training data, all the spans must be valid and non-overlapping, or an error will be thrown.

LocationValue
Doc.entsThe annotated spans. Tuple[Span]
Token.ent_iobAn enum encoding of the IOB part of the named entity tag. int
Token.ent_iob_The IOB part of the named entity tag. str
Token.ent_typeThe label part of the named entity tag (hash). int
Token.ent_type_The label part of the named entity tag. str

Config and implementation

The default config is defined by the pipeline component factory and describes how the component should be configured. You can override its settings via the config argument on nlp.add_pipe or in your config.cfg for training.

SettingDescription
phrase_matcher_attrOptional attribute name match on for the internal PhraseMatcher, e.g. LOWER to match on the lowercase token text. Defaults to None. Optional[Union[int, str]]
matcher_fuzzy_compare v3.5The fuzzy comparison method, passed on to the internal Matcher. Defaults to spacy.matcher.levenshtein.levenshtein_compare. Callable
validateWhether patterns should be validated (passed to the Matcher and PhraseMatcher). Defaults to False. bool
overwrite_entsIf existing entities are present, e.g. entities added by the model, overwrite them by matches if necessary. Defaults to False. bool
ent_id_sepSeparator used internally for entity IDs. Defaults to "||". str
scorerThe scoring method. Defaults to spacy.scorer.get_ner_prf. Optional[Callable]
explosion/spaCy/master/spacy/pipeline/entityruler.py

EntityRuler.__init__ method

Initialize the entity ruler. If patterns are supplied here, they need to be a list of dictionaries with a "label" and "pattern" key. A pattern can either be a token pattern (list) or a phrase pattern (string). For example: {"label": "ORG", "pattern": "Apple"}.

NameDescription
nlpThe shared nlp object to pass the vocab to the matchers and process phrase patterns. Language
name v3.0Instance name of the current pipeline component. Typically passed in automatically from the factory when the component is added. Used to disable the current entity ruler while creating phrase patterns with the nlp object. str
keyword-only
phrase_matcher_attrOptional attribute name match on for the internal PhraseMatcher, e.g. LOWER to match on the lowercase token text. Defaults to None. Optional[Union[int, str]]
matcher_fuzzy_compare v3.5The fuzzy comparison method, passed on to the internal Matcher. Defaults to spacy.matcher.levenshtein.levenshtein_compare. Callable
validateWhether patterns should be validated, passed to Matcher and PhraseMatcher as validate. Defaults to False. bool
overwrite_entsIf existing entities are present, e.g. entities added by the model, overwrite them by matches if necessary. Defaults to False. bool
ent_id_sepSeparator used internally for entity IDs. Defaults to "||". str
patternsOptional patterns to load in on initialization. Optional[List[Dict[str, Union[str, List[dict]]]]]
scorerThe scoring method. Defaults to spacy.scorer.get_ner_prf. Optional[Callable]

EntityRuler.initialize methodv3.0

Initialize the component with data and used before training to load in rules from a pattern file. This method is typically called by Language.initialize and lets you customize arguments it receives via the [initialize.components] block in the config.

NameDescription
get_examplesFunction that returns gold-standard annotations in the form of Example objects. Not used by the EntityRuler. Callable[[], Iterable[Example]]
keyword-only
nlpThe current nlp object. Defaults to None. Optional[Language]
patternsThe list of patterns. Defaults to None. Optional[Sequence[Dict[str, Union[str, List[Dict[str, Any]]]]]]

EntityRuler.__len__ method

The number of all patterns added to the entity ruler.

NameDescription

EntityRuler.__contains__ method

Whether a label is present in the patterns.

NameDescription
labelThe label to check. str

EntityRuler.__call__ method

Find matches in the Doc and add them to the doc.ents. Typically, this happens automatically after the component has been added to the pipeline using nlp.add_pipe. If the entity ruler was initialized with overwrite_ents=True, existing entities will be replaced if they overlap with the matches. When matches overlap in a Doc, the entity ruler prioritizes longer patterns over shorter, and if equal the match occurring first in the Doc is chosen.

NameDescription
docThe Doc object to process, e.g. the Doc in the pipeline. Doc

EntityRuler.add_patterns method

Add patterns to the entity ruler. A pattern can either be a token pattern (list of dicts) or a phrase pattern (string). For more details, see the usage guide on rule-based matching.

NameDescription
patternsThe patterns to add. List[Dict[str, Union[str, List[dict]]]]

EntityRuler.remove methodv3.2.1

Remove a pattern by its ID from the entity ruler. A ValueError is raised if the ID does not exist.

NameDescription
idThe ID of the pattern rule. str

EntityRuler.to_disk method

Save the entity ruler patterns to a directory. The patterns will be saved as newline-delimited JSON (JSONL). If a file with the suffix .jsonl is provided, only the patterns are saved as JSONL. If a directory name is provided, a patterns.jsonl and cfg file with the component configuration is exported.

NameDescription
pathA path to a JSONL file or directory, which will be created if it doesn’t exist. Paths may be either strings or Path-like objects. Union[str,Path]

EntityRuler.from_disk method

Load the entity ruler from a path. Expects either a file containing newline-delimited JSON (JSONL) with one entry per line, or a directory containing a patterns.jsonl file and a cfg file with the component configuration.

NameDescription
pathA path to a JSONL file or directory. Paths may be either strings or Path-like objects. Union[str,Path]

EntityRuler.to_bytes method

Serialize the entity ruler patterns to a bytestring.

NameDescription

EntityRuler.from_bytes method

Load the pipe from a bytestring. Modifies the object in place and returns it.

NameDescription
bytes_dataThe bytestring to load. bytes

EntityRuler.labels property

All labels present in the match patterns.

NameDescription

EntityRuler.ent_ids property

All entity IDs present in the id properties of the match patterns.

NameDescription

EntityRuler.patterns property

Get all patterns that were added to the entity ruler.

NameDescription

Attributes

NameDescription
matcherThe underlying matcher used to process token patterns. Matcher
phrase_matcherThe underlying phrase matcher used to process phrase patterns. PhraseMatcher
token_patternsThe token patterns present in the entity ruler, keyed by label. Dict[str, List[Dict[str, Union[str, List[dict]]]]
phrase_patternsThe phrase patterns present in the entity ruler, keyed by label. Dict[str, List[Doc]]