Pipeline

SpanRuler

classv3.3
String name:span_rulerTrainable:
Pipeline component for rule-based span and named entity recognition

The span ruler lets you add spans to Doc.spans and/or Doc.ents using token-based rules or exact phrase matches. For usage examples, see the docs on rule-based span matching.

Assigned Attributes

Matches will be saved to Doc.spans[spans_key] as a SpanGroup and/or to Doc.ents, where the annotation is saved in the Token.ent_type and Token.ent_iob fields.

LocationValue
Doc.spans[spans_key]The annotated spans. SpanGroup
Doc.entsThe annotated spans. Tuple[Span]
Token.ent_iobAn enum encoding of the IOB part of the named entity tag. int
Token.ent_iob_The IOB part of the named entity tag. str
Token.ent_typeThe label part of the named entity tag (hash). int
Token.ent_type_The label part of the named entity tag. str

Config and implementation

The default config is defined by the pipeline component factory and describes how the component should be configured. You can override its settings via the config argument on nlp.add_pipe or in your config.cfg.

SettingDescription
spans_keyThe spans key to save the spans under. If None, no spans are saved. Defaults to "ruler". Optional[str]
spans_filterThe optional method to filter spans before they are assigned to doc.spans. Defaults to None. Optional[Callable[[Iterable[Span], Iterable[Span]], List[Span]]]
annotate_entsWhether to save spans to doc.ents. Defaults to False. bool
ents_filterThe method to filter spans before they are assigned to doc.ents. Defaults to util.filter_chain_spans. Callable[[Iterable[Span], Iterable[Span]], List[Span]]
phrase_matcher_attrToken attribute to match on, passed to the internal PhraseMatcher as attr. Defaults to None. Optional[Union[int, str]]
matcher_fuzzy_compare v3.5The fuzzy comparison method, passed on to the internal Matcher. Defaults to spacy.matcher.levenshtein.levenshtein_compare. Callable
validateWhether patterns should be validated, passed to Matcher and PhraseMatcher as validate. Defaults to False. bool
overwriteWhether to remove any existing spans under Doc.spans[spans key] if spans_key is set, or to remove any ents under Doc.ents if annotate_ents is set. Defaults to True. bool
scorerThe scoring method. Defaults to Scorer.score_spans for Doc.spans[spans_key] with overlapping spans allowed. Optional[Callable]
explosion/spaCy/master/spacy/pipeline/span_ruler.py

SpanRuler.__init__ method

Initialize the span ruler. If patterns are supplied here, they need to be a list of dictionaries with a "label" and "pattern" key. A pattern can either be a token pattern (list) or a phrase pattern (string). For example: {"label": "ORG", "pattern": "Apple"}.

NameDescription
nlpThe shared nlp object to pass the vocab to the matchers and process phrase patterns. Language
nameInstance name of the current pipeline component. Typically passed in automatically from the factory when the component is added. Used to disable the current span ruler while creating phrase patterns with the nlp object. str
keyword-only
spans_keyThe spans key to save the spans under. If None, no spans are saved. Defaults to "ruler". Optional[str]
spans_filterThe optional method to filter spans before they are assigned to doc.spans. Defaults to None. Optional[Callable[[Iterable[Span], Iterable[Span]], List[Span]]]
annotate_entsWhether to save spans to doc.ents. Defaults to False. bool
ents_filterThe method to filter spans before they are assigned to doc.ents. Defaults to util.filter_chain_spans. Callable[[Iterable[Span], Iterable[Span]], List[Span]]
phrase_matcher_attrToken attribute to match on, passed to the internal PhraseMatcher as attr. Defaults to None. Optional[Union[int, str]]
matcher_fuzzy_compare v3.5The fuzzy comparison method, passed on to the internal Matcher. Defaults to spacy.matcher.levenshtein.levenshtein_compare. Callable
validateWhether patterns should be validated, passed to Matcher and PhraseMatcher as validate. Defaults to False. bool
overwriteWhether to remove any existing spans under Doc.spans[spans key] if spans_key is set, or to remove any ents under Doc.ents if annotate_ents is set. Defaults to True. bool
scorerThe scoring method. Defaults to Scorer.score_spans for Doc.spans[spans_key] with overlapping spans allowed. Optional[Callable]

SpanRuler.initialize method

Initialize the component with data and used before training to load in rules from a pattern file. This method is typically called by Language.initialize and lets you customize arguments it receives via the [initialize.components] block in the config. Any existing patterns are removed on initialization.

NameDescription
get_examplesFunction that returns gold-standard annotations in the form of Example objects. Not used by the SpanRuler. Callable[[], Iterable[Example]]
keyword-only
nlpThe current nlp object. Defaults to None. Optional[Language]
patternsThe list of patterns. Defaults to None. Optional[Sequence[Dict[str, Union[str, List[Dict[str, Any]]]]]]

SpanRuler.__len__ method

The number of all patterns added to the span ruler.

NameDescription

SpanRuler.__contains__ method

Whether a label is present in the patterns.

NameDescription
labelThe label to check. str

SpanRuler.__call__ method

Find matches in the Doc and add them to doc.spans[span_key] and/or doc.ents. Typically, this happens automatically after the component has been added to the pipeline using nlp.add_pipe. If the span ruler was initialized with overwrite=True, existing spans and entities will be removed.

NameDescription
docThe Doc object to process, e.g. the Doc in the pipeline. Doc

SpanRuler.add_patterns method

Add patterns to the span ruler. A pattern can either be a token pattern (list of dicts) or a phrase pattern (string). For more details, see the usage guide on rule-based matching.

NameDescription
patternsThe patterns to add. List[Dict[str, Union[str, List[dict]]]]

SpanRuler.remove method

Remove patterns by label from the span ruler. A ValueError is raised if the label does not exist in any patterns.

NameDescription
labelThe label of the pattern rule. str

SpanRuler.remove_by_id method

Remove patterns by ID from the span ruler. A ValueError is raised if the ID does not exist in any patterns.

NameDescription
pattern_idThe ID of the pattern rule. str

SpanRuler.clear method

Remove all patterns the span ruler.

SpanRuler.to_disk method

Save the span ruler patterns to a directory. The patterns will be saved as newline-delimited JSON (JSONL).

NameDescription
pathA path to a directory, which will be created if it doesn’t exist. Paths may be either strings or Path-like objects. Union[str,Path]

SpanRuler.from_disk method

Load the span ruler from a path.

NameDescription
pathA path to a directory. Paths may be either strings or Path-like objects. Union[str,Path]

SpanRuler.to_bytes method

Serialize the span ruler to a bytestring.

NameDescription

SpanRuler.from_bytes method

Load the pipe from a bytestring. Modifies the object in place and returns it.

NameDescription
bytes_dataThe bytestring to load. bytes

SpanRuler.labels property

All labels present in the match patterns.

NameDescription

SpanRuler.ids property

All IDs present in the id property of the match patterns.

NameDescription

SpanRuler.patterns property

All patterns that were added to the span ruler.

NameDescription

Attributes

NameDescription
keyThe spans key that spans are saved under. Optional[str]
matcherThe underlying matcher used to process token patterns. Matcher
phrase_matcherThe underlying phrase matcher used to process phrase patterns. PhraseMatcher