Perfectly Browned Thanks to Camera and Artificial Intelligence in the Oven
Reading time for this article: 9 minutes
Whether super crispy or delicately browned, our new ovens with integrated camera and artificial intelligence prepare pizzas, lasagna and rolls just the way you like them. Technical Project Manager Maximilian Wild and AI Product Owner Hans-Martin Eiter talk about the underlying technologies and how they work in this short interview.
When baking, roasting and grilling, the flavor factor "browning" is simply part of the process. Whether a dish should be particularly crispy or just delicately browned depends on personal taste. In order to satisfy these personal preferences in the oven, consumers often had no choice in the past but to continuously monitor their dish and remove it from the oven at the right moment. Thanks to an integrated camera and artificial intelligence, BSH's new oven series now prepares pizza, lasagna and rolls to perfection, completely automatically. Consumers simply select the desired degree of browning on a scale of one to five, and the oven program switches itself off when the dish is ready. In the following interview, Technical Project Manager Maximilian Wild and AI Product Owner Hans-Martin Eiter talk about the underlying technologies and how they work.
Maximilian, you and your team have integrated a camera in the oven. What is special about the camera solution?
Maximilian Wild: We are not the first home appliance manufacturer to have a camera in the oven, but our solution is truly unique in market comparison: We not only offer very high image quality, but also cover all heating modes. So, the camera also works in combination with integrated microwave, steam and pyrolysis - under extreme conditions, so to speak.
That sounds like a lot of technical challenges! What was the biggest one and how did you solve it?
Max: The biggest challenge was cooling the camera. It can tolerate a maximum of 85° Celsius, and during pyrolysis the oven is heated up to 500 °Celsius. To ensure that the camera is still sufficiently cooled and functions properly, we use a passive cooling air flow generated by the oven's cooling air motor. Since the oven has always needed to be cooled due to high temperatures, we were able to use the existing motor. However, the thin layer of housing in which the cooling air moves to the camera did not exist before. We developed this cooling air system completely from scratch. It was a fine art to build this system around the appliance and reconcile it with microwave and steam operation at the same time, but we mastered it!
And what about humidity and steam?
Max: As mentioned earlier, that's a big challenge as well. The camera is located at the top left of the oven, which is one of the hottest spots in the unit. Therefore, fortunately, the camera fogs up only briefly because the steam condenses in cooler places. It would be a problem if moisture penetrated the camera lens, though. That's why, with the help of countless iteration loops and endurance tests in cooperation with the endurance test lab, we equipped the camera with a special combination seal made of graphite and brass. This means that it remains tight and keeps an optimum view of the dishes in the oven.
The camera is an important essential for perfect browning results. Hans-Martin, how does the camera know when the result is right?
Hans-Martin Eiter: This is where software and artificial intelligence (AI) come into play. Let's take a pizza, for example. Consumers select the desired browning result on a five-level scale via the Home Connect app or directly on the oven's display. Once the baking program starts, the camera continuously monitors the pizza and takes a photo every ten seconds. We have developed a software algorithm to assign a browning level to each image in the series. Thanks to this artificial intelligence, the oven knows when the desired degree of browning has been reached and switches off automatically. This works for a wide range of homemade and ready-made pizzas with different toppings and doughs, rolls and lasagna - covering hundreds of variations.
Wow, that's a lot! How did you teach the artificial intelligence to do that?
Hans-Martin: Like humans, the system learns from experience and has to 'see' many images of food during the baking process. That's why, together with our development experts for baking applications, we took thousands of pictures of the baking process of a wide variety of foods in Traunreut, Germany, and evaluated the degree of browning in these pictures. In the process, we 'showed' the AI system not only the most common lasagna dishes, pizzas and rolls, but also a wide range of ingredients and shapes - from asparagus pizza to pretzels or frozen lasagna. Because the AI is aware of this wide range of training data, it can identify the appropriate browning level even for the most diverse pizza toppings and doughs. This means consumers can take their dishes out of the oven without constant visual inspection, just at the right moment when they are sufficiently browned. Another cool added value for the foodies among our consumers: They can share the high-resolution photos from the oven directly with others and whet their appetites!
And what's next?
Hans-Martin: Well, first of all, we presented the new feature at the EuroCucina trade fair in June and at the IFA in Berlin in early September 2022. It will be available in the connectable ovens from Bosch’s accent line and Siemens’ studioLine series starting in January 2023. I'm really excited to see whether our consumers will be as satisfied as the many test users who have already been able to test the feature intensively in advance. Of course, we will continue to work in the meantime and develop further AI-based functions for the different needs of our consumers. Similar to autonomous driving in cars, we are so far only at the beginning of the road towards sensor-based cooking automation.
