OCR at high level has 2 main tasks
Problems with OCR are
The recent trend in deep learning for NLP is “Attention Mechanism”. This method is closely related to the visual attention we see in human beings. When we are looking at the image we don’t focus on the overall image at once we focus on certain parts of the image with high resolution while perceiving the low resolution and adjusting the focal point over time.
Human attention allows us to focus on certain regions with “high resolution” (ear in the yellow box) while perceiving the “low resolution” (like snowy background and coat of the dog) and adjust the focal…
Entropy is the measure of the reduction in uncertainty.
Entropy came from Claude Shannon's information theory, where the goal is to send information from the sender to the recipient in an optimized way. We use bits to send information, a bit is either 0 or 1.
When we are using one bit to send a piece of information we are reducing the recipient's uncertainty by a factor of 2.
Suppose we have two types of weather Rainy & Sunny, And the forecast has predicted next day would be rainy, here we reduced the uncertainty by a factor of 2…
Intelligence is not to act when you are uncertain
There can be two types of errors associated with any kind of Machine learning or deep learning model
The overall error made by any model is a combination of the above two errors.
When we have done any lab experiment, the values measured after multiple trials will never be the same. Even with all similar input values output measurements will differ every time you run. This is what…
Neural structured learning is a framework used for training neural networks with structured signals. This can be applied to NLP, vision or any prediction problem in general(classification, Regression).
The structure can be explicitly given (like knowledge graphs in NLP) or structure can be generated on the fly while training (like creating adversarial examples with perturbations in the data).
Why is this helpful?
ONNX stands for an Open Neural Network Exchange is a way of easily porting models among different frameworks available like Pytorch, Tensorflow, Keras, Cafee2, CoreML.Most of these frameworks now support ONNX format.
Pytorch is the most preferred language of researchers for their experiments because of its pythonic way of writing code compared to TensorFlow. But when you have to deploy it to production Tensorflow has the best stack like Tensorflow Serving. Currently, there is no easy way to convert Tensorflow models to Pytorch. This is where ONNX shines.
Step 1 Convert your Pytorch model to ONNX format
from transformers import…
Dropout applied to regular neural network cannot be applied to RNN’s as it will hinder the RNN’s ability to retain the long term dependency.
To over come this Drop Connect is used:
Instead of operating on the RNN’s hidden states, one can regularize the network through restrictions on the recurrent matrices as well.This can be done through restricting the capacity of the matrix, can be applied without any modification to existing LSTM implementations. This weight-dropped LSTM applies recurrent regularisation through a DropConnect mask on the hidden-to-hidden recurrent weights. …
Lead AI Applied Research