Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [X] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [X] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [X] Is your initial contribution a single, squashed commit?

For code changes:

• [X] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [ ] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [X] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [X] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [X] Is your initial contribution a single, squashed commit?

For code changes:

• [X] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [X] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Added a patch that changes multiple items (Sorry That I did not fix 1 item at a time-I am a scientist not a software engineer):

1.) Added method to EventTrainer – train(DataIndexer indexer) –Then I added the method to AbstractEventTrainer. No other code needed to be changed. 2.) Created a new class – PluggableParmeters: Only AbstractTrainers had access to getXXXParam(Value,Default). So I pulled out this functionality into a separate class. Now both AbstractTrainers and AbstractDataIndexers can hold parameters. 3.) Refactored DataIndexer. This touched a lot of code. Added init(Map,Map) method and index(ObjectStream) method. Changed 1-Pass and 2-Pass DataIndexers. Everywhere 1-pass/2-pass indexers where created, I changed the constructor, and added the init and index methods. 4.) Changed GIS.doTrain(indexer) to use the parameters passed in the init method. 5.) QNTrainer – created a working init method. Changed the isValid method so it is not the init method.

#4 Make the GIS class work within the new training API

Addresses OPENNLP-1350

The MAIL_REGEX in UrlCharSSequenceNormalizer causes replaceAll(...) to become extremely costly when given an input string with a long sequence of characters from the first character set in the regex, but which ultimately fails to match the whole regex. This pull request fixes that, and also another detail:

Allow + in the local part, and disallow _ in the domain part. There are other characters that are allowed in the local part as well, but these are less common (https://en.wikipedia.org/wiki/Email_address).

The speedup for unfortunate input is achieved by adding a negative lookbehind with a single characters from the first character set. Currently, the replaceAll(" ") on a string of ~100K characters from the set [-_.0-9A-Za-z] runs in ~1minute on modern hardware; adding a negative lookbehind with one of the characters from that set reduces this to a few milliseconds, and is functionally equivalent. (Consider the current pattern and a match from position i to k. If the character at i-1 is in the character set, there would also be a match from i-1 to k, which would already be replaced.)

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [x] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [x] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [ ] Is your initial contribution a single, squashed commit?

  • Not yet, intentionally, in case we decide to trim some of the changes

For code changes:

• [x] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [ ] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Security Vulnerability Fix

This pull request fixes a Temporary File Information Disclosure Vulnerability, which existed in this project.

Preamble

The system temporary directory is shared between all users on most unix-like systems (not MacOS, or Windows). Thus, code interacting with the system temporary directory must be careful about file interactions in this directory, and must ensure that the correct file posix permissions are set.

This PR was generated because a call to File.createTempFile(..) was detected in this repository in a way that makes this project vulnerable to local information disclosure. With the default uname configuration, File.createTempFile(..) creates a file with the permissions -rw-r--r--. This means that any other user on the system can read the contents of this file.

Impact

Information in this file is visible to other local users, allowing a malicious actor co-resident on the same machine to view potentially sensitive files.

Other Examples

• CVE-2020-15250 - junit-team/junit
• CVE-2021-21364 - swagger-api/swagger-codegen
• CVE-2022-24823 - netty/netty
• CVE-2022-24823 - netty/netty

The Fix

The fix has been to convert the logic above to use the following API that was introduced in Java 1.7.

File tmpDir = Files.createTempFile("temp dir").toFile();

The API both creates the file securely, ie. with a random, non-conflicting name, with file permissions that only allow the currently executing user to read or write the contents of this file. By default, Files.createTempFile("temp dir") will create a file with the permissions -rw-------, which only allows the user that created the file to view/write the file contents.

:arrow_right: Vulnerability Disclosure :arrow_left:

:wave: Vulnerability disclosure is a super important part of the vulnerability handling process and should not be skipped! This may be completely new to you, and that's okay, I'm here to assist!

First question, do we need to perform vulnerability disclosure? It depends!

1. Is the vulnerable code only in tests or example code? No disclosure required!
2. Is the vulnerable code in code shipped to your end users? Vulnerability disclosure is probably required!

Vulnerability Disclosure How-To

You have a few options options to perform vulnerability disclosure. However, I'd like to suggest the following 2 options:

1. Request a CVE number from GitHub by creating a repository-level GitHub Security Advisory. This has the advantage that, if you provide sufficient information, GitHub will automatically generate Dependabot alerts for your downstream consumers, resolving this vulnerability more quickly.
2. Reach out to the team at Snyk to assist with CVE issuance. They can be reached at the Snyk's Disclosure Email.

Detecting this and Future Vulnerabilities

This vulnerability was automatically detected by GitHub's CodeQL using this CodeQL Query.

You can automatically detect future vulnerabilities like this by enabling the free (for open-source) GitHub Action.

I'm not an employee of GitHub, I'm simply an open-source security researcher.

Source

This contribution was automatically generated with an OpenRewrite refactoring recipe, which was lovingly hand crafted to bring this security fix to your repository.

The source code that generated this PR can be found here: SecureTempFileCreation

Opting-Out

If you'd like to opt-out of future automated security vulnerability fixes like this, please consider adding a file called .github/GH-ROBOTS.txt to your repository with the line:

User-agent: JLLeitschuh/security-research
Disallow: *

This bot will respect the ROBOTS.txt format for future contributions.

Alternatively, if this project is no longer actively maintained, consider archiving the repository.

CLA Requirements

This section is only relevant if your project requires contributors to sign a Contributor License Agreement (CLA) for external contributions.

It is unlikely that I'll be able to directly sign CLAs. However, all contributed commits are already automatically signed-off.

The meaning of a signoff depends on the project, but it typically certifies that committer has the rights to submit this work under the same license and agrees to a Developer Certificate of Origin (see https://developercertificate.org/ for more information).

- Git Commit Signoff documentation

If signing your organization's CLA is a strict-requirement for merging this contribution, please feel free to close this PR.

Sponsorship & Support

This contribution is sponsored by HUMAN Security Inc. and the new Dan Kaminsky Fellowship, a fellowship created to celebrate Dan's memory and legacy by funding open-source work that makes the world a better (and more secure) place.

This PR was generated by Moderne, a free-for-open source SaaS offering that uses format-preserving AST transformations to fix bugs, standardize code style, apply best practices, migrate library versions, and fix common security vulnerabilities at scale.

Tracking

All PR's generated as part of this fix are tracked here: https://github.com/JLLeitschuh/security-research/issues/18

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [X] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [X] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [X] Is your initial contribution a single, squashed commit?

For code changes:

• [X] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [X] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

I was working on a personal tutorial site for OpenNLP and till now I was trying to showcase some sample for running Sentence Detector API. Hope it would be helpful for community.

You can have a look at initial demo. http://my-ai-launcher.appspot.com/#/visualnlp/opennlp

Also Refer https://issues.apache.org/jira/secure/attachment/12872265/sentdetect.png

I am working on adding more API's and option for loading custom models. Further I will add support for training of customer models.

Please share your valuable feedback.

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [X] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [X] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [ ] Is your initial contribution a single, squashed commit?

For code changes:

• [X] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [X] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [x] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [x] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [x] Is your initial contribution a single, squashed commit?

For code changes:

• [x] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [x] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes: [* ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[*] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[*] Has your PR been rebased against the latest commit within the target branch (typically master)?

[*] Is your initial contribution a single, squashed commit?

For code changes: [ ] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder? [ ] Have you written or updated unit tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0? [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder? [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder? For documentation related changes: [* ] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Change

• adds missing JavaDoc
• improves existing documentation for clarity
• removes superfluous text
• adds 'final' modifier where useful and applicable
• adds 'Override' annotation where useful and applicable
• adds package-info.java file to entitylinker package
• fixes some typos

Tasks

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [x] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [x] Has your PR been rebased against the latest commit within the target branch (typically main)?

• [x] Is your initial contribution a single, squashed commit?

For code changes:

• [x] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [ ] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.

What does this PR do?

Tests are not failing on Windows anymore. This PR adds build support for Windows (latest = Server 2022) and adjusts test cases (if needed) to be run on GH actions CI.

• Adds CI for Windows (latest = Server 2022, similar to Windows 11) to be able to build / test with Windows
• Unifies how line endings are handled by git via .gitattributes as we have a checkstyle rule in place which prohibits the use of CRLF (checkstyle will fail a build on Windows if git isn't configured accordingly)
• Converts line endings from CRLF to LF (according to supplied .gitattributes, see https://docs.github.com/en/get-started/getting-started-with-git/configuring-git-to-handle-line-endings) via git add --renormalize .
• Workarounds https://github.com/junit-team/junit5/issues/2811 (@TempDir created folders cannot be deleted on Windows Server 2022)
• Removes Coveralls as there is no Jacoco support at the moment (so the step isn't useful anymore): https://github.com/coverallsapp/github-action/issues/22

Matrix build is now happy: https://github.com/rzo1/opennlp/actions/runs/3809019320

For all changes:

• [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [x] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [x] Has your PR been rebased against the latest commit within the target branch (typically main)?

• [x] Is your initial contribution a single, squashed commit?

For code changes:

• [x] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [x] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.

Change

• adds Internal annotation to code which has existing hints/notes on internal use
• improves some JavaDoc by linking to existing classes or interfaces
• fixes inconsistent JavaDoc in SentenceModelLoader
• adds 'final' modifier where useful and applicable
• adds 'Override' annotation where useful and applicable
• fixes some typos

Tasks

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [x] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [x] Has your PR been rebased against the latest commit within the target branch (typically main)?

• [x] Is your initial contribution a single, squashed commit?

For code changes:

• [x] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [ ] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [x] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [x] Has your PR been rebased against the latest commit within the target branch (typically main)?

• [x] Is your initial contribution a single, squashed commit?

For code changes:

• [x] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [x] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.

…other one

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [ ] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [ ] Is your initial contribution a single, squashed commit?

For code changes:

• [ ] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [ ] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.

Thank you for contributing to Apache OpenNLP.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

• [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• [x] Does your PR title start with OPENNLP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• [x] Has your PR been rebased against the latest commit within the target branch (typically master)?

• [x] Is your initial contribution a single, squashed commit?

For code changes:

• [x] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
• [x] Have you written or updated unit tests to verify your changes?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder?
• [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder?

For documentation related changes:

• [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.