SPARKNLP-835: ProtectedParam and ProtectedFeature #13797
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR enables
Param
s andFeature
s to be protected. Once set, they can't be changed again. This is useful for specific pretrained models, where some parameters should not be changed (as it could interfere with the functionality of the pretrained model).Protected
Feature
sProtected
Feature
s were already introduced in the PR #13777 but are finalized in this PR. An warning will be printed when trying to set a protected feature and it will not be set.Protected
Param
sProtected
Param
s are introduced with the traitHasProtectedParams
:https://github.com/DevinTDHa/spark-nlp/blob/0fdab1d0c6eb32125558b77f5d7e33adcbcd5346/src/main/scala/com/johnsnowlabs/nlp/HasProtectedParams.scala
It introduces an implicit conversion from a
Param
to aProtectedParam
. This can be done, by callingsetProtected()
on a regular Spark parameter. For Example:The conversion enables additional functionality while setting the value for the parameter. For the
set
function, it will check, whether or not the (now)ProtectedParam
is protected. If so, it will only allow for the parameter to be set once:spark-nlp/src/main/scala/com/johnsnowlabs/nlp/HasProtectedParams.scala
Lines 47 to 56 in 0fdab1d
Note: Default values do not count as set and can therefore be overridden
Motivation and Context
Some parameters should not be changed after initialization. This PR addresses this missing feature.
How Has This Been Tested?
Tests have been added. The behavior can be seen here:
spark-nlp/src/test/scala/com/johnsnowlabs/nlp/HasProtectedParamsTestSpec.scala
Lines 30 to 41 in 0fdab1d
TODO:
ProtectedParams
will need to override some additional functions.Types of changes
Checklist: