DateParser failing to detect dates

[csr03sridharan]

DateMatcher is not detecting dates as expected. in Spark NLP

Here 1/2 is identified as date in pyspark - Please check this.

Attached the below code.

Example: text = ["right over-the-needle catheter system 18 gauge;1 1/2 in length"]

class NLPPipeline:
def init(self, pipeline_stage_type):
self.pipeline_stage_type = pipeline_stage_type
self.documentAssembler = DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")

self.date_parser = DateMatcher() \
  .setInputCols("document") \
  .setOutputCol("date") \
  .setAnchorDateYear(1900) \
  .setDateFormat("yyyy/MM/dd")
self.pretrained_model_pipeline = self.pretrained_model_pipeline_runner()

def pretrained_model_pipeline_runner(self):
if self.pipeline_stage_type == 'DATE_ONLY':
pipeline = Pipeline().setStages([
self.documentAssembler,
self.date_parser])
return pipeline

[csr03sridharan]

the issue is actually with the word gauge, when we remove it from the sentence it does not detect dates

[wolliq]

Hello @csr03sridharan,

thank you for reporting. We detected a partial match in the relaxed parser:

• RuleMatch(18,relaxed days) coming from number 18 in the text
• RuleMatch(aug,relaxed months exclusive) aug coming from the word gauge
• default relaxed year which is the current. 2021

So when you remove the word "gauge" the "aug" subword is no more detected and the relaxed date is no more matched.

We will be fixing it asap to avoid this behavior, thank you

[maziyarpanahi]

Fixed in Spark NLP 3.3.3 release