Feature Transformation is the process of converting raw data which can be of Text, Image, Graph, Time series etc… into numerical feature (Vectors). So that we can perform all algebraic operation on it.

Text data usually consists of documents which can represent words, sentences or even paragraphs of free flowing text. The inherent unstructured (no neatly formatted data columns!) and noisy nature of textual data makes it harder for machine learning methods to directly work on raw text data. Hence, in this article, we will explore some of the most popular and effective strategies for transforming text data into feature…

What Is Natural Language Processing (NLP)?
Natural Language Processing (NLP) is a technique where text data like Mail, Social Media Posts, Web Page Content, SMS etc. is processed for extracting information from text data and use it in our computations and algorithm.

Text preprocessing is an essential step in building a Machine Learning model and depending on how well the data has been preprocessed, the results are seen.

In the vector space model, each word/term is an axis/dimension. The text/document is represented as a vector in the multi-dimensional space.
The number of unique words means the number of dimensions.

After reading this post you will know:

  1. What is Answer Selection Model
  2. Text preprocessing
  3. How to split data for Train and Test
  4. Vectorization of Preprocessed data
  5. Data visualization using Wordcloud
  6. What is embedding Layer
  7. Padding
  8. Model Selection
  9. LSTM Network
  10. Model Architecture
  11. Training and Testing model
  12. Building Flask Api and deployment of model

1. What is Answer Selection Model?

Given a question and a set of answers, answer selection is the task of identifying which of the answer is correct to the question. It is an important problem in natural language processing.
For Example : Given the question “Who established the Nobel Prize?”
and the following ans,
1. The Nobel Prize was established more than 100…

After reading this post you will know:

  1. what is Feature Extraction?
  2. What is Response code?
  3. Python code to compute response code
  4. Model Selection
  5. Why LightGBM is used?
  6. BayesianOptimization for LightGBM
  7. Training LightGBM model

1. What is Feature Extraction?

Feature Extraction, also known as feature creation, is the process of constructing new features from existing data to train a machine learning model. This step can be more important than the actual modeling because a machine learning algorithm only learns from the data we give it, and creating features that are relevant to a task is absolutely crucial. …

Prassena Kannan

Working as Machine Learning engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store