Can Artificial Intelligence understand and analyse the typical Indian thali— a plate comprising different food varieties, with mixed textures and overlapping ingredients?
The researchers at the Hyderabad-based International Institute of Information Technology (IIIT) say it can.
A team of scientists from the Centre for Visual Information Technology at IIIT-Hyderabad, using computer vision and artificial intelligence, is developing tools to understand a typical Indian thali comprising multiple dishes such as rice, dal, roti, chutney, curd and papad.
Understanding an Indian meal is far more complex than analysing a burger or a sandwich, because of the multiplicity of the ingredients with varying nutritional values.
Prof CV Jawahar, who has been leading the project said, “If you are given a full plate of typical Indian food that not only has multiple dishes, but mixed ones like rice topped with dal, a roti hidden under a papad.… how do you understand what is there on a plate and eventually its nutritional value?”
This problem is critical as most existing food-tracking apps are designed for discrete and standardised Western meals. Existing food-scanner and nutrition systems assume a fixed menu and stable recipes. Indian food breaks both assumptions.
Instead of retraining models, the IIIT-H team built a zero-shot system, which can recognise new food items without starting over. According to researchers, the system first identifies food regions without knowing exactly what the food item is. Then, instead of rigid classification, the system uses retrieval-based prototype matching. This approach makes the system scalable, flexible, and realistic for places like cafeterias and hospital messes.
The research began not in a restaurant but with a healthcare need. The request was raised particularly around monitoring nutrition for pregnant women, and that helped shape this work.
Accordingly, the research team built a zero-shot system. It first identifies regions on the plate and then matches them with visual examples, allowing it to work even when the menu changes overnight in cafeterias or hospital messes.
The current setup works with an overhead camera at a kiosk. “We want to extend it to an app-based system,” said Yash Arora, one of the researchers. To prepare for this, the team captured data from multiple angles, allowing food to be scanned from a phone camera – not just a fixed setup. The team, however, acknowledged that there are many practical issues still to be solved.
The research, which resulted in a paper titled ”What is there in an Indian thali” authored by Yash Arora and Aditya Arun under the guidance of Prof. Jawahar was presented at the 16th Indian Conference on Computer Vision, Graphics and Image Processing 2025.
Beyond individual plates, IIIT-H researchers are also building an “Indian food map” to capture how food varies across regions. Since food is deeply tied to culture, the idea is to visualise differences in ingredients, tastes and cooking styles across the country. This includes mapping raw materials, spice preferences and regional flavours. “Different parts of the country have different types of food, or sometimes similar dishes but different ways of cooking them,” remarks Prof Jawahar.
Also read: Telangana accounts for India's one in every five Typhoid deaths
Follow us
