Students from our Artificial Intelligence and Robotics (AIR) and Applied Artificial Intelligence (AAI) majors, after three months, completed their project, Robotaurant, for the module ZA-3201 Intelligent Systems Lab.
The team integrated three robots to automate the operation of a restaurant. NAO serves as the customer attendant, taking orders through natural conversation. An embedded large language model (LLM) agent transcribes customers’ speech using a speech-to-text engine and interprets their requests. The orders are then forwarded to Ned, which prepares the requested items and places them on a serving tray. Ned executes a state machine that manages the entire preparation process, from parsing orders and locating ingredients to assembling the requested items. Once the order is ready, Stretch receives a notification from Ned, picks up the serving tray, and autonomously delivers it to the customer using ROS-based navigation.
Despite having much to learn and overcoming many messy trial-and-error moments along the way, the team successfully developed an initial working prototype. They showcased the final product to a group of teachers from Sultan Hassanal Bolkiah Institute of Education (SHBIE).
