Add Natural Language Processing (Word Cloud)

Natural Language Processing

When the "Requires Language Processing" option on a Text field is enabled (and a Language Processing Destination field is set) and users change the data in that field, the system will extract all of the important keywords from the entered text and populate the Language Processing Destination field.

The process of extracting the important keywords is called "lemmatization", where our Natural Language Processing engine will analyze the text to categorize each word based on its grammatical context, remove words from specific grammatical categories, and then reduce the remaining words down to their "lemma" or base/dictionary form. The resulting keywords, which are deemed to be of most "analytical value" are populated into the Language Processing Destination field for use in Word Cloud visualizations.

Multi-lingual Systems

Natural Language Processing in SAI360 only works on English text – however if you have a Google API Key with translation APIs enabled, SAI360 can automatically translate text from other languages to English before performing Natural Language Processing.

This means every record, regardless of the entered language, can have the generated keywords in English – which is extremely valuable for Word Cloud visualizations.

To accommodate languages other than English you need to

• Set up an Account for Google Translate and either retrieve an API key or link it to your existing Google API key.
• If you have a new Google API Key, enter this key in the System page of the Global Settings.
  Should you already have an API key (for Google Maps) no action is required.
• Once set up, the web application will translate non-English entries (using the Google API) to generate English 'keywords' for the Word Cloud.

Without an API key, non-English entries will be ignored for the purpose of generating the Word Cloud.

Setting up a Natural Language Processing

In SAI360, Language Processing can only be defined for Text fields, but not for Memo fields.

The setup process is as follows:

• For each Text field you want Language Processing performed on, create and save another Text field which will be used as the Language Processing Destination Field.

  As a guideline, use the same Maximum Size as the Source Field.

  Also, this Destination field can be on a related Component, as long as it is a 1-1 Relationship.
  Choosing a setup like this can in some instances improve performance.

• Select the Source field, and
  • tick Language Processing Required
  • on the tab Language Processing Configuration, select the Language Processing Destination Field.
• Save and Publish your Changes.
• To populate these fields for existing records, it is necessary to design and run an Analytics Rule.
  Please contact your SAI360 representative for assistance.

  (An automated process to perform this step will be available in a future release).