Topic modeling is a branch of natural language processing that is used to extract topics from a corpus (or collection of articles), in a (typically) unsupervised manner. This is done by extracting the patterns of word clusters and frequencies of words in the document.
The extracted topics allow one to identify similar articles based on the topics covered by them. This also gives the ability to perform the search for content by topic rather than a keyword search. The main advantage of using Topic Modelling is when you are having a large corpus of documents and you wish to know what the documents talk about i.e. what type of information is present in the document.
There are many topic modeling techniques available but LDA (Latent Dirichlet Allocation) is the most popular one. In LDA, the word “latent” indicates the hidden topics present in the data while the word “Dirichlet” is a form of distribution. Please note, “Dirichlet” distribution is different from “Normal” distribution, as in “Normal Distribution” represent the data in real numbers whereas “Dirichlet Distribution” represents the data such that the plotted data sums up to 1.
Following are some practical use cases for Topic Modelling:
- Audit of contracts for regulation compliance
- Sentiment Analysis
- Classification of text based on Topics
- Understanding scientific publications
- Text Summarization
- Recommendation systems can be used to group services ultimately resulting in the more appropriate matching of users and services
We at smartData Enterprises (I) Ltd, have been working for our prestigious clients to extract the relevant information from various contracts and audit them to confirm their compliance against the available regulations and had developed recommendation engines.
Share on
