Recipient Organization
AGROFOCAL TECHNOLOGIES, INC
3598 COUR DE JEUNE
SAN JOSE,CA 951484306
Performing Department
(N/A)
Non Technical Summary
Agriculture is fundamental to our existence. As our world grows, agriculture productivity must increase to keep pace without reducing profits or increasing environmental impact. This requires continuous optimization of agriculture processes. However, we cannot optimize something that we cannot measure accurately. Crop monitoring is an essential activity that provides such measurements on crop status.Crop monitoring involves checking on the crops for any plant health issues, for estimating the produce, for planning labor deployment, or for other farming decisions. Traditionally this is done manually with experienced farmers walking their fields and observing the crops. Clearly with many hundreds of acres of farms, it is impossible to do this manually; only a small fraction of the fields can be surveyed in this fashion. Several companies are working to solve this problem using technology, but their solutions don't fit well with prevalent farming practices. Manyusedrones, aircrafts, or satellite images, which workwell forproviding broad overview of farms, but fail to look under the canopy and observe issues that require per-plant pictures. Many require images of the fields to be uploadedonto a central serverbefore they can be processed.The step of uploading images to a central server is time-consuming and, in many cases, requires manual oversight to complete. This approach prevents crop monitoring from being real-time. In addition, the amount of data that is required to be uploaded limits the size of the farm area that can be surveyed. Many companies provide crop monitoring as a service, where they dothe task of image collection and processing and then provide a report. In such cases, their monitoringschedule may or may not match with what is required by the growers. If a grower needs to do the monitoring every day orweek for every farm they own, it is usually extremely difficult for suchcompaniesto provide that level of service at a reasonable cost.Agrofocal's goal is to build a real-time crop monitoring system that is easy to use, is affordable, and fits seamlessly with the existing farming operations. The crop monitoring system can be mounted on any moving vehicle going through the farm. This allows the camera to get up-close, under-the-canopy view of the crop. The entire process of collecting the images and operating on those images to extract the insights is done on the vehicle itself in real-time, which eliminates the time-consuming process of uploading the images to a central server for processing. The insights are available and ready to be viewed on an app as soon as the image collection is done. Since in this technology the image collection and the processing happen together without any manual intervention, it can be completely owned and operated by the grower themselves, without requiring the involvement of the company providing the technology. This way the grower can mount the system on their vehicle and do the monitoring as many times as they require. They can also choose to mount this system on a vehicle that would be going through the farm to perform some other tasks. This can enable them to do the monitoring while performing the other tasks, such as weeding, pest-control, spraying, etc., thereby saving time and money.The goal of this project is to study and improvethe technologies that are foundational to Agrofocal's crop monitoring system. We have following three key objectives.(1) Device a robust and fault-tolerant protocol for image streaming between the camera and on-vehicle computer so that there are no disruptions in image stream during operation in rugged environment.(2) Develop agriculture-specific artifical intelligence algorithms that are better suited for recognizingplants issues, fruits, and other agriculture features as compared to existing algorithms which were primarily developed for non-agriculture uses. (3) Buildcamera harnesses suited for mounting cameras for rugged field operations that can dampen the vibrations felt on farm vehicles so that the cameras can capture clear imagesduring bumpy field operations. All together, this work will ensure that the crop monitoring system can function with accuracy when mounted on a vehicle moving through farms without havingto changehow the vehicle is driven or any other existing work practices.Agrofocal's crop monitoring solution will provideuseful insights in real-time, will be easy to use, and will fit seamlessly with existing farming operations. It will enable farmers to make informed decisions based on objective crop measurements. The insights provided will drive efficiencies in agriculture that will increase production, reduce costs, and lessen environmental impact. These efficiencies will positively change agriculture to the benefit of everyone.
Animal Health Component
40%
Research Effort Categories
Basic
10%
Applied
40%
Developmental
50%
Goals / Objectives
The goal of thisproject is to design a real-time crop monitoring system that is easy to use, is affordable, and fits seamlessly with the existing farming operations. The system will provideuseful insights that will enable farmers to drive efficiencies in agriculture that will increase production, reduce costs, and lessen environmental impact. These efficiencies will positively change agriculture to the benefit of everyone.Our crop monitoring system can be mounted on any moving vehicle going through the farm. The entire process of collecting the images and operating on those images to extract the insights will be done on the vehicle itself in real-time, eliminating the time-consuming process of uploading images to a central server for processing. The system be completely owned and operated by the grower themselves, without requiring the involvement of the company providing the technology.This affordable, easy to use technology provides the farmer a flexible monitoring schedule, reduced labor costs and crop yield prediction. Our technology will overcomethe limitations of offline monitoring and will benefit growers, shippers, crop-advisorcompanies, and industry consortiums by providing affordable, real-time monitoring of crops to predict yield and identify area of concern within the fields.The Phase I objectives of this project are as follows:Objective 1: Computer Vision: Seamless streaming of 4K or higher resolution images from camera to the compute box.Research and develop computer vision algorithms for seamlessly stream 4K images at 15 fps from thecamera to the compute box. Making the protocol fault tolerant so that dropped frames, due to ruggedand in-motion nature of the operations, are handled gracefully. The counting algorithms aredeveloped that can accurately count objects in streaming images considering intermittent occlusion.Objective 2: AI Models: Create real-time Artificial Intelligence (AI) Object detection model tunedfor various crop types.Research and develop AI models that are optimized for each crop type. This will involve studyingwhat neural network architectures will work well for the feature sizes and motion blur encounteredin agriculture imaging. Experiment with methods of labeling objects in images in presence of densefoliage, occlusion, different lighting conditions to help train AI models that work well withuncontrolled environments.Objective 3: Mechanical: Tractor attachments for vibration isolation and dampening and temperature control.Research and develop camera mounts that can isolate or dampen the kind of vibrations felt on farm vehicles, protect the camera from overheating during operations on hot sunny days, and generally rugged for the field operation.
Project Methods
The methods to be applied for the project are described below for each objective.Objective 1: Computer Vision: Seamless streaming of 4K or higher resolution images from camera to the compute box.Determine ways to measure "congestion" in image traffic between camera and the receiver. Prototype different server-client streaming protocols between camera and the receiver to see which one works better for our use case in terms of images/sec and "congestion".Study how "congestion" variesin our use case and device a flow control mechanism to prevent frame drops in presence of excessive congestion.Objective 2: AI Models: Create real-time Artificial Intelligence (AI) Object detection model tuned for various crop types.Real-time agriculture-specific AI object detection models.Train objection detection networks such as SSD, YoloV5, RetinaNet on the agriculture data set. Compare accuracy results and inference latency across them.Investigate areas where the architecture of these models can be modified, given the type of objects that will be detected, to improve the accuracy and latency of these models. (Note: this investigation can be a lot of work and doing such an indepth study on architecture for all networks may be out of scope for the time duration of the project. We may focus our efforts on one networkwhose base architecture shows better accuracy results and whose latency fits within the real-time needs).Crop-specific data set collected under realistic conditions. The AI model is as good as the data used to train it. We need images for different crop types collected from a moving vehicle, so that it has natural motion blur, at different times of day, at different times of the year, at different stages of growth, etc. These images collected and then appropriately labelled will create an invaluable resource for training accurate AI models. The method used will be:Mount camera on a vehicle, similar to how it will be during crop monitoring, and collect images for different crop types, at different times of the season, under different weather conditions, and at times of day.Get inputs from the crop advisors on what features on the image to get labelled and get them labelled accordingly.Real-time algorithms for counting plant features in moving images with intermittent occlusion.Study computer vision algorithms that can track motion for vehicle moving taking into account the speeds of motion, distance of plants, and camera FPS.Study different counting heuristics on moving images.Get hand count of fruits from the field to validate the counts calculated from the videos.Objective 3: Mechanical: Tractor attachments for vibration isolation and dampening and temperature control.Use vibration measurement instruments to determine the frequencies and amplitude of vibrations felt by the camera mounted on farm vehicles during operation.Use the above vibration data to research existing vibration isolators that may help dampen the shocks or work with a mechanical engineer to design the required isolator.Customize the camera software to compensate for the movement that cannot be removed with mechanical isolator.Study camera casing to protect it from sunlight and from passing branches.