Software apps and online services
Our robot car (named Chsel) is designed to right wall follow in an experimental setup. As Chsel follows the wall it pans across the area in search of fruit. When fruit is detected the robot car will stop right-wall following and pursue the fruit. Upon reaching the fruit, a servo actuated gripper mechanism will close around the fruit. Once the fruit is obtained, the robot car will then search for home base and return the fruit home.
A state machine was implemented on the TI F28379 Launch Pad to drive the logic behind the robot car’s actions. Eight different states were used: right-wall follow, front-wall turn, room panning for fruit, fruit pursuit, fruit collection, pan room for home base, home base pursuit, and fruit drop off. Meanwhile a camera is wired to an orange pi with another state machine with states for searching for fruit and searching for home. The launch pad and orange pi communicate via a serial port to coordinate the camera efforts with the driving and collecting efforts.
When Chsel first turns on, the default state on the launch pad is to right-wall follow. In order to right-wall follow infrared (IR) sensors are used in a feedback loop. One sensor is placed on the right side of the wall to detect when the vehicle is too close or too far from the right wall. The second IR sensor is placed on the front edge of the car. The feedback loop takes an input voltage from both sensors and compares them to a reference voltage. A filter is applied to both sensors to reduce any noise and occasional false readings that may occur. For the right sensor, the filtered voltage received from the sensor is subtracted from a reference voltage and then multiplied by a gain. This is the turn command that the robot receives. When the front sensor receives an input voltage suggesting that the robot is too close to the front wall, the robot changes to the front-wall turn state and a turn command telling the robot to turn about 90 degrees is given to the robot.
While the launch pad is in the right-wall follow and front wall turn states, the camera is constantly searching for fruit. The orange pi uses open cv to help identify the fruits. The fruit detection algorithm looks at the color and shape of the object to determine if it is a fruit. However, the camera only has a limited field of view and may miss fruit if it only looks directly in front of it while wall following. Because of this problem, another state was implemented telling the robot to pan the room every couple of seconds to detect any fruit that may have originally been outside of its scope. When fruit is detected by the orange pi-camera system, the orange pi sends a signal to the launch pad indicating that a fruit has been seen and relays the fruit size and position information. This will cause the launch pad to enter another state where the turn and speed commands are driven by the fruit size and position.
Once the orange pi determines that the robot is close enough to use the servo-gripper mechanism to grab the fruit, the orange pi will send another signal to the launch pad telling it to grab the fruit. During this state the robot will turn the servo, causing the gripper mechanism to encapsulate the fruit.
After grabbing the fruit, a servo attached to the camera is told to actuate up so that the robot has a larger field of view. Then the robot spins until it detects the home base (a blue shirt). Once the camera detects the blue shirt, it will send another signal to the launchpad telling it that the home base has been detected and giving the home base size and location details. Again the launchpad will use this information to adjust the turn and speed commands so that the robot drives to the blue home base. After arriving to the home base, the servo-gripper mechanism releases the fruit and the robot will back up. Then the camera will angle downward to detect fruit again and the process will repeat.Demo Video
The following video shows the robot car successfully gathering various fruits in the test course.
In the collection video, the robot successfully collects all fruits and returns them to the home area. In this case, home is defined as a few feet away from the shirt. At the 2:20 mark in the video, the robot briefly lost connection with the server and briefly did not behave properly, but by 2:50, connection was restored. The processing on the orange pi can also be shown on the screen record video, which shows the camera detecting a fruit, modifying the coordinates sent to the launch pad, and then looking for the home location.
Once the fruit was collected, the robot looks for blue objects. Initially it found a blue object outside the course, but as it approached the wall, it realized this was no longer visible and began looking again. In this case, the robot then found the blue shirt, the actual home.Video Explaining Parts Used and Connection to Launch Pad
The next video outlines the various sensors and actuators used in the project and how they are connected to the TI LaunchXL F28379 processor.Conclusion
Through the use of a TI LaunchXL F28379 processor, Orange Pi, camera, two IR sensors, and two servos, Chsel was able to successfully collect apples, oranges, pears, and lemons.