Hidde Vermeulen | Cameron Mensah-Hoek | Chane Engelbrecht | Tobias Bielderman | Mathias Col
For this project, our team of 5 focused on developing an automated picking system for small mosaic tile kits. Pixel Corner challenged us to develop a reliable automated alternative system.
Currently, around 100 kits are packed by hand each day, with workers counting and weighing 40–300 tiles across 26 different types, colours and sizes. This takes a lot of time and can lead to mistakes, such as missing tiles or using the wrong colour or size, so the main challenge is keeping the error rate below 5%. Pixel
The Solution
The PXC Tile Sorter uses a gantry robot (driven by GRBL), a camera and a custom end effector to pick up individual mosaic tiles and put together complete tile kits. Since the tiles are randomly spread across the bed, can touch each other and are sometimes chipped or broken, one of the biggest challenges is recognising which tiles are good and picking them up accurately.
To solve this, we developed a vision system using a 1440p camera and OpenCV. The camera separates the tiles from the backlit sections and measures their shapes to determine whether they are good, damaged or not square. The colours are recognised using samples that can be taught through the web interface. By using the edges of each section, the system converts pixels into millimetres, allowing the gantry to know the exact position of each tile.
Once the tile has been identified and its position is known, the end effector picks it up using one of its five suction cups. A servo rotates the cups into the correct position, while a pneumatic piston lowers them onto the tile. When the cups are holding the tiles, the head moves them one by one into the funnel of the kit.
A second controller makes sure the sections stay supplied with tiles. The three buffer motors release new tiles into their sections, while the single vibrating plate helps spread the tiles out when too few good tiles are visible. If too many damaged tiles build up in a section, the robot picks them out first and drops them in a separate location.
The colours, quantities and number of kits can all be set through the web interface.
Software and integration
One Python program controls the whole machine. It reads the camera, makes decisions and sends commands to the gantry and the two Arduino Mega controllers. A built-in web server also makes it possible to view everything through a browser.
The vision software converts each tile’s position in the image into millimetres from the centre of its section. The machine then adds the measured distance between the camera and the pick tool, as well as a small correction for each suction cup. This makes sure the correct cup lands on the tile. Before picking up a tile, the camera moves above it and corrects its position until it is accurate to within a fraction of a millimetre.
All hardware is connected through USB. The gantry receives standard G-code commands through GRBL, while the end effector and tile distributor use short text commands, such as “pick with cup 3” or “release buffer 1”. Each controller confirms when a command has been completed. Both controllers also send their status several times per second and stop their motors automatically if they lose communication with the PC.
The software also handles errors during normal operation, rather than stopping the entire machine. At startup, it checks whether each controller is running the correct software and tests the suction cups. During a run, if a tile has moved or disappeared, the machine skips it and continues with the next one. If a feeding command is interrupted, it is simply tried again. When a section stays empty after several releases, the machine asks the operator to refill the buffer.
The operator can pause, resume or stop the machine at any time through the web interface. When the machine is paused, the progress is saved so it can continue from where it left off. The web interface also displays the live camera view and the current status of the machine.