The series of blog posts under this theme researches on how robot’s interactions can bring the true grounding in a human-robot dialogue. I am discussing my understandings of these works. Today’s paper Collaborative Effort towards Common Ground in Situated Human-Robot Dialogue, experiments different robot interactions during a dialogue and measures its impact in bringing the common grounding.
While reading the survey paper on Trends in Integration of Vision and Language Research I came across the following broad research areas in NLP and CV domains, jotting them here for future reference.
Majority of the previous works in task-oriented systems, question generator used dialogue history to generate the next question in the conversation. So far, the understanding was dialogue history, along with answers aided in anticipating the best probable future question. There is also a paper on answer-driven visual estimation to improve the task success rate. However there is not much analysis on how the dialogue history is impacting the oracle’s responses. How oracle can use dialog history? This is the theme for today’s blog post. The blog post discusses about the paper, Region under Discussion for visual dialog
True to its title, the paper, Hallucinating Entities in GuessWhat?! brings out how existing question generators hallucinating entities while conversing with the oracle. Hallucination referes to generating words that are not matching with source data. And these hallucinations creating a domino effect in the generated dialogues and degrading the overall quality of the system. Such systems are harmful to deploy in real-world as it persons with viewing disabilities value correctness over the fine-grained descriptions.
Taking the leap forward from the previous post, what are the other startegies humans follow? It is found that, we, humans usually follow confirmation-driven strategies when searching for an information (or) learning a new language. Inspired by this theory, a new approach is proposed to converse and locate object(s) in GuessWhat game. The blog post discusses about the paper Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy.
One of the problems that was discussed in previous blog posts was, Question Generator is not aware of the objects in the image. Hence the questions start with categories and the dialog flows if ‘Yes’ is encountered along the line. Also during the trails, it was found humans usually start with entity based questions. How to equip question generator with entity information?
In the previous blog posts, the question generator almost always starting with entity/category questions and majority of other questions are spatial. However these kind of questions hit a block, if multiple objects share attributes, or the visual scene is complex. Also the Guesser targets a single object and does not consider the relationship among objects.
In this blog post, I am discussing my understandings from the paper Visual Dialogue State Tracking for Question Generation
In this blog post, I am discussing my understandings from the paper Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat
While there are numerous research directions in task-oriented systems, I am picking the question generator for my reading task, as it is the crucial component in continuing the dialogue. Please check these posts (introductory paper, available datasets) for a overview of task-oriented systems.
Who am I? is a popular guessing game where the players use yes/no questions to guess the identity of person/animal/place. Visual object discovery mimics the game with two players (oracle and questioner) through dialog. In this blog post, lets dive into the introductory paper where GuessWhat?! dataset was introduced.
In continuation with the previous post, here are some more datasets that are useful to work with Visual Dialogue Systems.
The dataset was first introduced in the paper: A Visually-grounded First-person Dialogue Dataset with Verbal and Non-verbal Responses
Predominantly for the task-oriented visual dialogue systems, the GuessWhat dataset has been in use. However, with the rise in popularity, new datasets have been proposed periodically to further up the game. This post(is part-1, part-2 here) discusses the present-day datasets for these systems.
Task-oriented visual dialogue systems aim to identify an object from a given complex visual scene, through a series of yes/no questions. The core idea is to enable machines to have natural conversations with humans.
Big thanks to Prof Dr. Neeldhara, for starting EXPORT (EXPress yOuR Thoughts) challenge with public accountability. This is the nudge I need after months of procrastination to update my tech blog.
Indian Diaspora is rich in languages. According to a 2018 survey, 9 out of 10 new internet users will be Indian language users. Moreover, close to 70% of people consume content in local languages. With the rise in demand, enthusiasts, freelancers and organisations started focusing on developing NLP tools for the Indian languages.
This post aims in sharing the existing set of NLP tools and their usage
Some of the datasets I came across while browsing is shared below.
NER For South and South East Asian Langauges
Tab delimited Bi-lingual Pairs
Kaggle - Telugu NLP Dataset
COVID-19 Resources from Facebook and Google
Man ki Bath Parallel Corpus
Parallel Corpus via CrowdSourcing for 6 Indian Languages
HuggingFace Datasets for 11 Indian Languages
Multilingual Bible Parallel Corpus
Open Parallel Corpus
From Keon’s Github
Finally got around to starting a blog on my #NLProc journey. Powered by Jekyll.
I would like to use this space to share my learnings, experiments, thoughts and opinions on #NLProc stuff.