Class LanguageDetectorME
- All Implemented Interfaces:
Serializable,LanguageDetector
LanguageDetector.
This will process the entire string when called with
predictLanguage(CharSequence) or
predictLanguages(CharSequence).
If you want this to stop early, use probingPredictLanguages(CharSequence)
or probingPredictLanguages(CharSequence, LanguageDetectorConfig).
When run in probing mode, this starts at the beginning of the char sequence
and runs language detection on chunks of text. If the end of the
string is reached or there are LanguageDetectorConfig.getMinConsecImprovements()
consecutive predictions for the best language and the confidence
increases over those last predictions and if the difference
in confidence between the highest confidence language
and the second highest confidence language is greater than
LanguageDetectorConfig.getMinDiff(), the language detector will
stop and report the results.
The authors wish to thank Ken Krugler and Yalder} for the inspiration for many of the design components of this detector.
- See Also:
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionString[]predictLanguage(CharSequence content) Predicts theLanguagefor the fullcontentlength.Language[]predictLanguages(CharSequence content) Predicts thelanguagesfor the fullcontentlength.probingPredictLanguages(CharSequence content) This will stop processing early if the stopping criteria specified inLanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIGare met.probingPredictLanguages(CharSequence content, LanguageDetectorConfig config) This will stop processing early if the stopping criteria specified inLanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIGare met.static LanguageDetectorModeltrain(ObjectStream<LanguageSample> samples, TrainingParameters mlParams, LanguageDetectorFactory factory) Starts a training of aLanguageDetectorModelwith the given parameters.
-
Constructor Details
-
LanguageDetectorME
Initializes an instance with a specificLanguageDetectorModel. Default feature generation is used.- Parameters:
model- theLanguageDetectorModelto be used.
-
-
Method Details
-
predictLanguages
Description copied from interface:LanguageDetectorPredicts thelanguagesfor the fullcontentlength.- Specified by:
predictLanguagesin interfaceLanguageDetector- Parameters:
content- The textual content to detect potentiallanguagesfrom.- Returns:
- the predicted languages
-
predictLanguage
Description copied from interface:LanguageDetectorPredicts theLanguagefor the fullcontentlength.- Specified by:
predictLanguagein interfaceLanguageDetector- Parameters:
content- The textual content to detect potentiallanguagesfrom.- Returns:
- the language with the highest confidence
-
getSupportedLanguages
- Specified by:
getSupportedLanguagesin interfaceLanguageDetector- Returns:
- Retrieves an array of language (codes) that are supported by a
LanguageDetector.
-
probingPredictLanguages
This will stop processing early if the stopping criteria specified inLanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIGare met.- Parameters:
content- content to be processed- Returns:
- A computed
ProbingLanguageDetectionResult.
-
probingPredictLanguages
public ProbingLanguageDetectionResult probingPredictLanguages(CharSequence content, LanguageDetectorConfig config) This will stop processing early if the stopping criteria specified inLanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIGare met.- Parameters:
content- The textual content to process.config- TheLanguageDetectorConfigto customize detection.- Returns:
- A computed
ProbingLanguageDetectionResult.
-
train
public static LanguageDetectorModel train(ObjectStream<LanguageSample> samples, TrainingParameters mlParams, LanguageDetectorFactory factory) throws IOException Starts a training of aLanguageDetectorModelwith the given parameters.- Parameters:
samples- TheObjectStreamofLanguageSampleused as input for training.mlParams- TheTrainingParametersfor the context of the training.factory- TheLanguageDetectorFactoryfor creating related objects defined viamlParams.- Returns:
- A valid, trained
LanguageDetectorModelinstance. - Throws:
IOException- Thrown if IO errors occurred.
-