Smart IoT Kitchen Scale for Accurate and Low-Effort Nutrition Tracking

No one likes food tracking. It’s tedious, hard to get right, and overall annoying. People want to eat, not play a guessing game. Unfortunately, for diabetics, estimating carbohydrates is routine whenever they want to eat, as the required amount of insulin is directly proportional to the amount of carbohydrates consumed.

Several applications promise precise carbohydrate estimation. Most require the user to take an image of their food, which is then analyzed by a model to return an estimate of the carbohydrate, protein, and fat contents. Some even use Lidar sensors on modern phones. However, these apps often struggle with the geometry of tableware. A similar-looking pile of pasta can greatly differ in mass depending on whether it is on a plate or in a bowl. Additionally, many dishes have layers of different ingredients. For example, with pasta, tomato sauce, and parmesan cheese, the model cannot discern if there is pasta under the sauce.

One day, I realized these issues could be resolved if the tech adapted to how people naturally assemble their food:

1. They get a plate.
2. Add ingredient by ingredient.
3. Serve & enjoy.

The key is the second step. People usually put together their meal ingredient by ingredient. Typically, pasta first, then tomato sauce, and finally cheese. It’s a discrete process. Instead of estimating the nutritional content of the entire meal, I can do it for each ingredient as it is added. All I would need to do is measure the weight change and identify the ingredient.

The goal of this project was to build a proof of concept (POC) based on this idea and compare its accuracy in macro estimates and convenience against existing apps.

Project Overview

Main Requirements To ensure the Smart IoT Kitchen Scale meets the needs of diabetics and anyone interested in accurate nutrition tracking, the following requirements were established:

• Accurate Nutritional Information: The system must determine accurate meal and ingredient-level nutritional information to ensure precise carbohydrate, protein, and fat tracking.
• Internet Accessibility:* The images and nutritional information must be accessible over the internet, allowing users to review their data remotely.
• Seamless Integration: The technology should blend into the kitchen environment and follow the pre-existing cooking process to avoid disrupting meal preparation.
• Minimal Additional Effort: Using the scale should require less than 10 seconds of additional effort to ensure convenience and ease of use.

Additional Constraints Considering practical and financial constraints, the project also adhered to the following guidelines:

• Cost-Effective Hardware: The total cost of the hardware components should not exceed $50 to make the solution affordable and accessible.
• Shared-Apartment Friendly: The system should be designed to avoid causing inconvenience to roommates, ensuring it is suitable for shared living spaces.

System Workflow The system's operation follows a straightforward process to track nutritional information accurately:

1. Weight Measurement and Imaging: Each time an ingredient is added, the system measures the change in weight and takes a picture.
2. Ingredient Identification: The pictures taken before and after adding the ingredient are used to identify what was added.
3. Nutritional Lookup: The system looks up the macronutrients for the identified ingredient.
4. Macronutrient Calculation: It calculates the macronutrients for the specific amount of the added ingredient.
5. Total Macronutrient Calculation: The system then calculates the total macronutrients of the entire dish.
6. User Interface: Finally, the nutritional information is made accessible through a user interface, allowing users to review their meal's nutritional content easily.

System Components The Smart IoT Kitchen Scale comprises three main components, each playing a crucial role in the system:

• Smart Scale: A Raspberry Pi combined with a camera and a load cell + analog-to-digital converter, responsible for measuring weight and capturing images.
• Backend: A Django-based backend with a PostgreSQL database, Celery message queue running in Redis, and OpenAI API integration for ingredient identification.
• Frontend: Currently, a simple Django Admin Dashboard is used to display the nutritional information to the user.

Implementation Details

Here is an overview of the system in more detail:

This project description only provides a high level overview of the entire project. For detials about the implementation, challenges I faced, and the solutions I created, check out my two dedicated posts:

• Hardware
• Backend

Results & Future Enhancements

The system works. I have recorded 44 meals with 220 ingredients, and it only takes a few seconds. Plus, none of my roommates have complained about the contraption in the shared kitchen. I call this a win.

However, the macro tracking is still annoying. There were occasions when I skipped the tracking and guessed the carbs. Dogfooding ("dogfooding is the practice of using one's own products or services." - wikipedia (opens in a new tab)) is brutal but important and helped identify issues with this V0 of the system:

Wiser's Calmness is great, but can't forget about Nielsen's Usability Heuristics.

The system was very unobtrusive. When not in use, it could be stored under the cabinets and was easy to deploy when needed. There were only two inputs: one to create and complete meals, and another to take pictures of the food. However, there was no output on the scale itself. There was no feedback. I couldn’t see the status of the previously created meal or if I had already clicked complete. I never knew if the picture was successfully taken or if the food was in frame. The only way to check the system status was to open the terminal on my laptop and SSH into the Raspberry Pi or the server dashboard.

A simple solution would be to add a small screen (even a 16x2 text display) to show the total weight of the food, if there is an open meal, or how many ingredients were added.

Another issue was error prevention. Automatically setting the attribute to complete after a certain amount of time (e.g., 1 hour) could address the problem of forgetting to click the complete button.

I also had to think hard about which button did what more than once. Adding metaphors or icons to the buttons would make them more intuitive, addressing Nielsen’s heuristic of recognition rather than recall.

Conclusion

The Smart IoT Kitchen Scale has proven to be a practical solution for accurate and low-effort nutrition tracking, especially for those managing type 1 diabetes. By measuring ingredients as they are added and calculating their nutritional content, this system offers a more reliable and user-friendly alternative to existing applications.

Key takeaways from this project include the importance of user feedback mechanisms and error prevention in creating a seamless user experience. The next steps involve improving the system’s usability by adding a display for real-time feedback and enhancing error prevention measures.

If you have built something similar or are interested in building upon this project, feel free to reach out. Let’s collaborate and make nutrition tracking even easier and more accurate.