Students: Mike van Amsterdam – Wouter van Leeuwen – Alexander Ramp
Ficus plants
Ficus trees come in all shapes and sizes. With hundreds of known species that range from trees, to vines and the potted plants found in living rooms. Most species are found in tropical climates, so potted ficus plants are cultivated in greenhouses.
Current situation
The cultivation of ficus plants is a difficult process to automate. The current situation is that some species of ficus trees tend to grow in a bush-like shape if grown without support. These species normally use other trees or bushes for support and thus in a greenhouse this needs to be artificially added to the pot. Farmers use small bamboo and wooden sticks of varying height to reproduce this support, with help of small clips to bind the main stems of the plants to the stick.
The binding (or clipping) process is always done manually, plants are placed on a conveyor belt and one labourer clips around five hundred plants per hour. For this process it’s very important that no side-twigs or leaves get caught up in the clip, just the main stem(s). If done incorrectly the plants final shape will not be proportional.
Annually Zwethlande kwekerij produces around 500.000 plants that need to be clipped, up to five times for each plant. They also produce ficus plants that don’t require support.
Assignment
The assignment given by Zwethlande kwekerij is to automate the binding of the ficus lyrata to a stick without damaging the plant.
To achieve this we set some objectives for the project:
– Find a way to detect the stem and the bamboo stick
– Find a way to bring them together
– Find a way to bind them together
– Position the camera in an optimal position
– Detect if plants are too small or impossible to bind properly
Our solution
Our solution consists of four subsystems:
– The rotation system;
– The detection system;
– The binding system;
– The operating system.
The rotation system
This sub system is used to rotate the plants in front of the camera. By doing this we can check the plants from more than one side to find a binding solution. This will eventually increase our success rate.
The detection system
For detection we use a light sensor to detect the incoming plant and a camera to get an image of the plant. The image is further processed in Halcon. Our Halcon script scans the plant from the top to the bottom and when it finds a binding solution it sends it to our robot system.
The binding system
We used an existing system, the attalink binding plier to bind the stem and stick together. We automated it by mounting a pneumatic cylinder on it. We added two grabbing hooks to this solution to bring the stem and stick together into the binding plier to further complete the solution.
The operating system
The heart of our solution is the UR10, complimented by a PLC which we use as a controlling interface. The UR10 runs a script which asks the Halcon program for a solution and acts on it. When no solution is sent to the UR by the Halcon program the script recognises it and makes the plant rotate. Then it requests another solution and that loops until a solution is found or if the maximum number of attempts has been made.
Major project decisions
Detection of stem and stick
A key obstacle in the project is how to determine where the stem and stick are. How do you recognize the features from one, or more, pictures? How do you setup your lighting for the background and plant?
For the complicated solution we used a vision program called Halcon. It allows us to fetch a lot of data from the image with just a simple webcam, which is mounted on the robot arm. There were a number of other possibilities for the detection system, one of them is 3D vision. This technology allows to see depth in the images and might have increased reliability. However with our very limited experience at the start of this project this would have made the detection considerably more difficult. We chose to look for solutions within the 2D environment.
Bringing stem and stick together
To bring the stem and stick together we came up with a solution that uses two hooks, actuated by a pneumatic cylinder. The hooks close in on the stem and stick and when the cylinder is actuated the stem and stick are grabbed by the hooks and pulled into each other. Other possible solutions would increasingly complicate the detection system and were deemed non-viable for the project due to time limitations.
Binding the stem and stick
Another obstacle that had to be overcome is binding the stem and stick together. The current situation makes use of clips designed to be handled by human hands. The problem with such a solution is not necessarily the handling, rather the supply. The solution used In the end of arm tool is a modified Attalink tying tool, actuated by a cylinder. It uses rope and aluminium reels to tie the stem and stick together. Rope is supplied in 100m rolls and aluminium in small rolls. Before reloading anything the combination ties around 1.000 plants.
Conclusion
The main goal of binding the stems to the sticks with use of machine vision has been achieved. But due to the big tool size the amount of plants we actually can bound is very limited. To increase the amount of plants which can be bound we have to design an smaller binding tool, also the tool has to be more plant friendly. So there is less chance of damaging the leaves. If this is achieved the Halcon program can be rewritten with the new tool size so it will see more opportunities to bind.
Another aspect that needs to be improved is that we need to add a third dimension to the image we use in the Halcon program. With a 3D image there are more opportunities to bind such as when the stick and stem are exactly in line with the lens of the camera or when a leaf is behind the stem and stick. At this moment the Halcon program did not see that as a solution.