GetPatternsFromDataMultiClass: NullPointerException with unknown cause

[aggarcia3]

Invoking GetPatternsFromDataMultiClass<SurfacePattern>.run(properties) on some corpora ends up always throwing a NullPointerException for the same reason. However, the processed named entity label when the exception is thrown is not the same between executions nor the first label that is processed, so I suspect some kind of race condition. Nevertheless, my suspicion doesn't seem to be the actual cause, because invoking the GetPatternsFromDataMultiClass<SurfacePattern>.run(properties) method in a synchronized block doesn't help (it is called from several threads in my code), neither reducing the numThreads property to 1.

In particular, the null pointer exception is caused by the setEn variable in the ScorePhrases.getSentences method being null. The stack trace of the exception is as follows:

java.lang.NullPointerException
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.ScorePhrases.getSentences(ScorePhrases.java:347)
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.ScorePhrases.applyPats(ScorePhrases.java:393)
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.ScorePhrases.learnNewPhrasesPrivate(ScorePhrases.java:567)
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.ScorePhrases.learnNewPhrases(ScorePhrases.java:172)
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.iterateExtractApply4Label(GetPatternsFromDataMultiClass.java:2307)
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.iterateExtractApply(GetPatternsFromDataMultiClass.java:2110)
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.runNineYards(GetPatternsFromDataMultiClass.java:3310)
	at stanford.corenlp@3.9.2/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.run(GetPatternsFromDataMultiClass.java:3290)
	at [my code]

Can someone point me to the right direction to fix this problem? It's urgent to me. Any help would be appreciated.

[AngledLuffa]

This isn't enough information to debug the issue.

[aggarcia3]

What more information can I provide to help? The input properties file?

[AngledLuffa]

Sure, and data files, and what you're actually doing to trigger this bug. That may very well have been enough information for the original author to diagnose, but everyone who worked on that project has since graduated and moved on.

[aggarcia3]

Thank you for the information. I hope we can find a solution to this problem even though the people involved with this code is no longer around here.

The properties I'm passing to the run method are roughly these: https://pastebin.com/tuAXyis4. Due to the design of my application, some properties are generated dynamically and then put in the properties object, namely: file, fileFormat, seedWordsFiles and patternsWordsDir. I've double checked that the dynamic properties are assigned sane values. In particular, the file property contains the absolute path to a serialized Map<String, DataInstance> object, and fileFormat is set to "ser". The rationale for this choice is to have more control about the NLP pipeline applied to the input and improve performance for my use case.

Interestingly, I've realized that, even after the exception is thrown, some learnedwords.txt output files contain partial results.

About the data files, apart from those referred to in the properties file, I'm afraid I can only provide examples of serialized sentence file I pass in the "file" property, and the seed words files. The whole corpora of documents is pretty big (a ~150 MiB SQLite database), and I have concerns about distributing it.

About what I'm doing to trigger the bug, it reduces to running that method with the described input. I'm able to consistently reproduce it in every run of the application.

[AngledLuffa]

Our hope is to produce a new release next week. I understand your hesitancy to share data. However, if you're able to share with us a prescription for reproducing the error, there's a chance we'll be able to fix it before the new release. If not, honestly it won't even be looked at until after next week.

[aggarcia3]

I'm developing a minimal toy project to share with you that is able to reproduce the issue on the master branch codebase. During that, I've accidentally found a cause of IndexOutOfBoundsException, and made a PR for it: #1025.

I've also rolled a quick and dirty fix that just skips execution of some code if variables are null to a hopefully temporary repository: https://github.com/aggarcia3/CoreNLP/blob/master/src/edu/stanford/nlp/patterns/ScorePhrases.java#L337. It seems to work fine; at least, the execution of the named entity extraction process continues without throwing more exceptions.

I'll post an update here when I'm able to reproduce this issue with the toy project, or not.

[aggarcia3]

Here's a toy project to reproduce issues I put together: CoreNLP issue 1024.zip. I didn't manage to reproduce this exact issue within this toy project yet, which further hints to some kind of race condition, but still I managed to consistently reproduce an issue, at least on my hardware: if you run it with mvn test, two tests should run successfully.

Both tests run exactly the same code with exactly the same input data and properties, which are included in the project. The only difference is the "numThreads" property, which when set to "1" works fine, but when set to "4" (the number of CPU cores of my system) a RuntimeException is thrown. The output of the tests is as follows:

[main] INFO edu.stanford.nlp.tagger.maxent.MaxentTagger - Loading POS tagger from https://github.com/aggarcia3/corenlp-models/raw/master/english-left3words-distsim.tagger ... done [1.8 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from https://github.com/aggarcia3/corenlp-models/raw/master/english.all.3class.distsim.crf.ser.gz ... done [4.7 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from https://github.com/aggarcia3/corenlp-models/raw/master/english.muc.7class.distsim.crf.ser.gz ... done [2.1 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from https://github.com/aggarcia3/corenlp-models/raw/master/english.conll.4class.distsim.crf.ser.gz ... done [2.0 sec].
[main] INFO minimaldebug - Number of seed words for label VACCINE_PREVENTABLE_DISEASE is 74
Reading english words from https://github.com/stanfordnlp/CoreNLP/raw/master/data/edu/stanford/nlp/patterns/surface/stopwords.txt
Size of othersemantic class variables is 0
Created index with size 0. Don't worry if it's zero and you are using batch process sents.
[main] INFO extremedebug - Matched phrases freq is {}
matched phrases for VACCINE_PREVENTABLE_DISEASE is {}
[main] INFO extremedebug - Matched phrases freq is {}
[main] INFO minimaldebug - Iterating 10 times.
[main] INFO minimaldebug - Saving the learned words for label VACCINE_PREVENTABLE_DISEASE in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE/learnedwords.txt
[main] INFO minimaldebug - Saving the learned patterns for label VACCINE_PREVENTABLE_DISEASE in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE/learnedpatterns.txt
[main] INFO minimaldebug - 

################################ Iteration 1 ##############################
[main] INFO minimaldebug - 
###Learning for label VACCINE_PREVENTABLE_DISEASE ######


java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException:  Why does the token not have the class class edu.stanford.nlp.patterns.PatternsAnnotations$PatternLabel1 set? Existing classes ;class edu.stanford.nlp.ling.CoreAnnotations$ValueAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$TextAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$OriginalTextAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$CharacterOffsetBeginAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$CharacterOffsetEndAnnotation:2;class edu.stanford.nlp.ling.CoreAnnotations$IsNewlineAnnotation:false;class edu.stanford.nlp.ling.CoreAnnotations$TokenBeginAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$TokenEndAnnotation:1;class edu.stanford.nlp.ling.CoreAnnotations$IndexAnnotation:1;class edu.stanford.nlp.ling.CoreAnnotations$SentenceIndexAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$PartOfSpeechAnnotation:SYM;class edu.stanford.nlp.ling.CoreAnnotations$LemmaAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation:O;class edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagProbsAnnotation:{O=0.999969383866168};class edu.stanford.nlp.ling.CoreAnnotations$CoarseNamedEntityTagAnnotation:O
	at edu.stanford.nlp.patterns.surface.CreatePatterns.getAllPatterns(CreatePatterns.java:85)
	at edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.processSents(GetPatternsFromDataMultiClass.java:1169)
	at edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.getPatterns(GetPatternsFromDataMultiClass.java:1208)
	at edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.iterateExtractApply4Label(GetPatternsFromDataMultiClass.java:2292)
	at edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.iterateExtractApply(GetPatternsFromDataMultiClass.java:2110)
	at edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.runNineYards(GetPatternsFromDataMultiClass.java:3310)
	at edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.run(GetPatternsFromDataMultiClass.java:3290)
	at com.github.aggarcia3.issue.corenlp.Issue1024Test.lambda$0(Issue1024Test.java:116)
	at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:55)
	at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:37)
	at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:2952)
	at com.github.aggarcia3.issue.corenlp.Issue1024Test.testIssue(Issue1024Test.java:125)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:81)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:198)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:138)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$2(TestTemplateTestDescriptor.java:106)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:442)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:106)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:137)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:89)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:542)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:770)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:464)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException:  Why does the token not have the class class edu.stanford.nlp.patterns.PatternsAnnotations$PatternLabel1 set? Existing classes ;class edu.stanford.nlp.ling.CoreAnnotations$ValueAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$TextAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$OriginalTextAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$CharacterOffsetBeginAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$CharacterOffsetEndAnnotation:2;class edu.stanford.nlp.ling.CoreAnnotations$IsNewlineAnnotation:false;class edu.stanford.nlp.ling.CoreAnnotations$TokenBeginAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$TokenEndAnnotation:1;class edu.stanford.nlp.ling.CoreAnnotations$IndexAnnotation:1;class edu.stanford.nlp.ling.CoreAnnotations$SentenceIndexAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$PartOfSpeechAnnotation:SYM;class edu.stanford.nlp.ling.CoreAnnotations$LemmaAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation:O;class edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagProbsAnnotation:{O=0.999969383866168};class edu.stanford.nlp.ling.CoreAnnotations$CoarseNamedEntityTagAnnotation:O
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at edu.stanford.nlp.patterns.surface.CreatePatterns.getAllPatterns(CreatePatterns.java:81)
	... 127 more
Caused by: java.lang.RuntimeException:  Why does the token not have the class class edu.stanford.nlp.patterns.PatternsAnnotations$PatternLabel1 set? Existing classes ;class edu.stanford.nlp.ling.CoreAnnotations$ValueAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$TextAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$OriginalTextAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$CharacterOffsetBeginAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$CharacterOffsetEndAnnotation:2;class edu.stanford.nlp.ling.CoreAnnotations$IsNewlineAnnotation:false;class edu.stanford.nlp.ling.CoreAnnotations$TokenBeginAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$TokenEndAnnotation:1;class edu.stanford.nlp.ling.CoreAnnotations$IndexAnnotation:1;class edu.stanford.nlp.ling.CoreAnnotations$SentenceIndexAnnotation:0;class edu.stanford.nlp.ling.CoreAnnotations$PartOfSpeechAnnotation:SYM;class edu.stanford.nlp.ling.CoreAnnotations$LemmaAnnotation:**;class edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation:O;class edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagProbsAnnotation:{O=0.999969383866168};class edu.stanford.nlp.ling.CoreAnnotations$CoarseNamedEntityTagAnnotation:O
	at edu.stanford.nlp.patterns.surface.SurfacePatternFactory.getContextTokenStr(SurfacePatternFactory.java:451)
	at edu.stanford.nlp.patterns.surface.SurfacePatternFactory.getContext(SurfacePatternFactory.java:195)
	at edu.stanford.nlp.patterns.surface.SurfacePatternFactory.getPatternsAroundTokens(SurfacePatternFactory.java:514)
	at edu.stanford.nlp.patterns.PatternFactory.getPatternsAroundTokens(PatternFactory.java:97)
	at edu.stanford.nlp.patterns.surface.CreatePatterns$CreatePatternsThread.call(CreatePatterns.java:135)
	at edu.stanford.nlp.patterns.surface.CreatePatterns$CreatePatternsThread.call(CreatePatterns.java:105)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
[main] INFO edu.stanford.nlp.tagger.maxent.MaxentTagger - Loading POS tagger from https://github.com/aggarcia3/corenlp-models/raw/master/english-left3words-distsim.tagger ... done [1.3 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from https://github.com/aggarcia3/corenlp-models/raw/master/english.all.3class.distsim.crf.ser.gz ... done [2.6 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from https://github.com/aggarcia3/corenlp-models/raw/master/english.muc.7class.distsim.crf.ser.gz ... done [1.3 sec].
[main] INFO edu.stanford.nlp.ie.AbstractSequenceClassifier - Loading classifier from https://github.com/aggarcia3/corenlp-models/raw/master/english.conll.4class.distsim.crf.ser.gz ... done [1.5 sec].
[main] INFO minimaldebug - Number of seed words for label VACCINE_PREVENTABLE_DISEASE is 74
Reading english words from https://github.com/stanfordnlp/CoreNLP/raw/master/data/edu/stanford/nlp/patterns/surface/stopwords.txt
Size of othersemantic class variables is 0
Created index with size 0. Don't worry if it's zero and you are using batch process sents.
[main] INFO extremedebug - Matched phrases freq is {measles=1.0}matched phrases for VACCINE_PREVENTABLE_DISEASE is {measles=1.0}

[main] INFO extremedebug - Matched phrases freq is {}
[main] INFO minimaldebug - Iterating 10 times.
[main] INFO minimaldebug - Saving the learned words for label VACCINE_PREVENTABLE_DISEASE in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE/learnedwords.txt
[main] INFO minimaldebug - Saving the learned patterns for label VACCINE_PREVENTABLE_DISEASE in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE/learnedpatterns.txt
[main] INFO minimaldebug - 

################################ Iteration 1 ##############################
[main] INFO minimaldebug - 
###Learning for label VACCINE_PREVENTABLE_DISEASE ######
shuffled 2 sentences and selecting 2 sentences per thread
[main] INFO minimaldebug - 

## Selected Patterns for VACCINE_PREVENTABLE_DISEASE##
[main] INFO minimaldebug - (?$term [{ner:O}]{1,3}) [{processedtextannotation:"without"}] [{processedtextannotation:"complication"}]:1
[main] INFO minimaldebug - [{processedtextannotation:"you"}] [{processedtextannotation:"survive"}]  (?$term [{ner:O}]{1,3}):1
[main] INFO minimaldebug - Saving output in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE
count for word U.S. is 0.0
ignoreWordsAll contains word U.S. is false
[main] INFO minimaldebug - 

## Selected Words for VACCINE_PREVENTABLE_DISEASE : 
[main] INFO minimaldebug - Saving output in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE
[main] INFO minimaldebug - 

################################ Iteration 2 ##############################
[main] INFO minimaldebug - 
###Learning for label VACCINE_PREVENTABLE_DISEASE ######
shuffled 2 sentences and selecting 2 sentences per thread
[main] INFO minimaldebug - 

## Selected Patterns for VACCINE_PREVENTABLE_DISEASE##
[main] INFO minimaldebug - Saving output in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE
count for word U.S. is 0.0
ignoreWordsAll contains word U.S. is false
[main] INFO minimaldebug - 

## Selected Words for VACCINE_PREVENTABLE_DISEASE : 
[main] INFO minimaldebug - Saving output in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/output/identifier/VACCINE_PREVENTABLE_DISEASE
[main] INFO minimaldebug - No patterns learned for all labels. Ending iterations.
[main] INFO minimaldebug - Writing justification files


All patterns learned:
VACCINE_PREVENTABLE_DISEASE:
Iteration 0
(?$term [{ner:O}]{1,3}) [{processedtextannotation:"without"}] [{processedtextannotation:"complication"}]
[{processedtextannotation:"you"}] [{processedtextannotation:"survive"}]  (?$term [{ner:O}]{1,3})
Iteration 1



All words learned:

Label VACCINE_PREVENTABLE_DISEASE

[main] INFO CoreNLP - Saving output in /media/discoduro/Documentos/Estudios/TFG/corenlp-issue-1024/identifier/model/

Here it is the test report generated by the Surefire plugin, with more verbose information about the execution environment: TEST-com.github.aggarcia3.issue.corenlp.Issue1024Test.xml.gz

I'll continue working on this toy project to see if I can reproduce the original issue in the same way, but it appears to be a kind of heisenbug...

[AngledLuffa]

Maybe I'm being dense, but I don't see any source files in here and `mvn test` didn't work (although admittedly we have an ancient version of maven installed) Can you send just the source file and data file? We'll run it without worrying about using maven

[aggarcia3]

Sorry about that; I made the .zip form a terminal and forgot to include the recursive flag for the program I used, so subdirectory contents weren't included. Here's the fixed archive: CoreNLP issue 1024.zip

Just for the record, I'm using Maven 3.6.3 for running the tests. The project is atypical in the sense that it only contains test code, with no actual release code, but it should run fine nevertheless. The source code is in corenlp-issue-1024/src/test/java/com/github/aggarcia3/issue/corenlp/Issue1024Test.java. The data files (docsents.ser, spied.properties and VACCINE_PREVENTABLE_DISEASE.txt) are in the archive, inside the corenlp-issue-1024 folder. If necessary, I guess I could provide only the source code and data files, but I'm afraid that would make the executions less reproducible, as the same versions of the needed dependencies should be manually included in the classpath instead.

[AngledLuffa]

Well, I found one bug which was very clearly affecting your test example. More threads than number of items would mean nothing gets processed. Perhaps there is a similar effect elsewhere in the system.

…

On Sun, Mar 22, 2020 at 8:09 AM Alejandro González García < ***@***.***> wrote: Sorry about that; I made the .zip form a terminal and forgot to include the recursive flag for the program I used, so subdirectory contents weren't included. Here's the fixed archive: CoreNLP issue 1024.zip <https://github.com/stanfordnlp/CoreNLP/files/4365467/CoreNLP.issue.1024.zip> Just for the record, I'm using Maven 3.6.3 for running the tests. The project is atypical in the sense that it only contains test code, with no actual release code, but it should run fine nevertheless. The source code is in corenlp-issue-1024/src/test/java/com/github/aggarcia3/issue/corenlp/Issue1024Test.java. The data files (docsents.ser, spied.properties and VACCINE_PREVENTABLE_DISEASE.txt) are in the archive, inside the corenlp-issue-1024 folder. If necessary, I guess I could provide only the source code and data files, but I'm afraid that would make the executions less reproducible, as the same versions of the needed dependencies should be manually included in the classpath instead. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1024 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2AYWKJEJTEF4NMIQHYJO3RIYS2FANCNFSM4LQVS6UA> .

[aggarcia3]

Thank you very much for looking into this and commiting that fix. I find your hyphotesis about the cause of this kind of issues reasonable. However, these days I've been trying to reproduce the original issue of this topic in a toy project without success. I've also faced other similar NullPointerException, but luckily they are infrequent enough that I can pretend they don't happen. I guess I need specific kinds of inputs, maybe in a particular order, to trigger these edge cases, but I don't really know what kind of combinations trigger these problems. I'm thinking on adding some debug logging to my application to correlate the exception with the particular input document. I'll try that as soon as possible, although I'll be busy this week. Of course, I'm open to any further suggestion or comment you might have for diagnosing this issue better. I'd like to help making CoreNLP better :) El mié., 25 de marzo de 2020 0:55, John Bauer <notifications@github.com> escribió:

…

Well, I found one bug which was very clearly affecting your test example. More threads than number of items would mean nothing gets processed. Perhaps there is a similar effect elsewhere in the system. On Sun, Mar 22, 2020 at 8:09 AM Alejandro González García < ***@***.***> wrote: > Sorry about that; I made the .zip form a terminal and forgot to include > the recursive flag for the program I used, so subdirectory contents weren't > included. Here's the fixed archive: CoreNLP issue 1024.zip > < https://github.com/stanfordnlp/CoreNLP/files/4365467/CoreNLP.issue.1024.zip > > > Just for the record, I'm using Maven 3.6.3 for running the tests. The > project is atypical in the sense that it only contains test code, with no > actual release code, but it should run fine nevertheless. The source code > is in > corenlp-issue-1024/src/test/java/com/github/aggarcia3/issue/corenlp/Issue1024Test.java. > The data files (docsents.ser, spied.properties and > VACCINE_PREVENTABLE_DISEASE.txt) are in the archive, inside the > corenlp-issue-1024 folder. If necessary, I guess I could provide only the > source code and data files, but I'm afraid that would make the executions > less reproducible, as the same versions of the needed dependencies should > be manually included in the classpath instead. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < #1024 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AA2AYWKJEJTEF4NMIQHYJO3RIYS2FANCNFSM4LQVS6UA > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1024 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTRPX2AZ4S3CUVT2ZJWGPDRJFB6TANCNFSM4LQVS6UA> .

[AngledLuffa]

Thanks again for contributing this test. I converted it to a format compatible with our current test suite, fixed the errors, and checked it in. ... ah, crap. just realized I did not properly change the --author. Sorry about that. What I found was that it wouldn't properly label text if the threads used was more than the sentences to label. If you had documents which had that happen, then maybe this fixed your issue. If not, I can continue debugging it if you provide some more examples of something that causes it to fail. If some of the data needed to trigger the error is proprietary, you can always send it to me personally and I'll destroy it after debugging.

[aggarcia3]

Nice work! I appreciate your efforts towards resolving this issue. I'm working with relatively short inputs, so it makes sense these exceptions happen to me. I will run a full test in the final application with the latest commits ASAP and let you know any stability differences. El jue., 26 de marzo de 2020 21:35, John Bauer <notifications@github.com> escribió:

…

Thanks again for contributing this test. I converted it to a format compatible with our current test suite, fixed the errors, and checked it in. ... ah, crap. just realized I did not properly change the --author. Sorry about that. What I found was that it wouldn't properly label text if the threads used was more than the sentences to label. If you had documents which had that happen, then maybe this fixed your issue. If not, I can continue debugging it if you provide some more examples of something that causes it to fail. If some of the data needed to trigger the error is proprietary, you can always send it to me personally and I'll destroy it after debugging. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1024 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTRPX7ZCD2JVNRA2ZVECF3RJO4AJANCNFSM4LQVS6UA> .

[AngledLuffa]

Was this sufficient? Wondering if we can close the issue

[aggarcia3]

I took a break these days, sorry for the delay.

Anyway, I have good news: all NullPointerException are gone! 🎉 Instead, now I get a much rarer ArithmeticException, whose stacktrace is as follows:

java.lang.ArithmeticException: / by zero
	at stanford.corenlp@3.9.2-CUSTOM/edu.stanford.nlp.patterns.surface.CreatePatterns.getAllPatterns(CreatePatterns.java:48)
	at stanford.corenlp@3.9.2-CUSTOM/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.processSents(GetPatternsFromDataMultiClass.java:1173)
	at stanford.corenlp@3.9.2-CUSTOM/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.getPatterns(GetPatternsFromDataMultiClass.java:1212)
	at stanford.corenlp@3.9.2-CUSTOM/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.iterateExtractApply4Label(GetPatternsFromDataMultiClass.java:2291)
	at stanford.corenlp@3.9.2-CUSTOM/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.iterateExtractApply(GetPatternsFromDataMultiClass.java:2109)
	at stanford.corenlp@3.9.2-CUSTOM/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.runNineYards(GetPatternsFromDataMultiClass.java:3309)
	at stanford.corenlp@3.9.2-CUSTOM/edu.stanford.nlp.patterns.GetPatternsFromDataMultiClass.run(GetPatternsFromDataMultiClass.java:3289)
	at es.uvigo.esei.sing.textproc.step.corenlpentityextraction@1.0.0-SNAPSHOT/es.uvigo.esei.sing.textproc.step.corenlpentityextraction.CoreNLPEntityExtractionProcessingStep.lambda$run$6(CoreNLPEntityExtractionProcessingStep.java:291)

I believe the underlying cause of this exception is some non-empty assumption not being held due to some input document with number of threads combination. Do you need any further information for debugging this (hopefully) last issue?