How John Deere Is Using AI and Machine Vision To Improve Agricultural Outcomes

As farming becomes as concentrated as many other technologically driven industries, the fusion of information and outcomes becomes ever more crucial to success. JL

Bernard Marr reports in Forbes:

Smart cameras powered by computer vision distinguish between healthy and unhealthy crops as machinery passes through the field. This system allows targeted bursts of chemicals to be directed precisely where they are needed, at individual plants, hence the 80 to 90% reduction in herbicide use. Deep neural networks analyze the quality of grain and make adjustments to the operating parameters of the machinery on-the-fly. More cameras monitor stalks, leaves, and cobs as they are ejected from the rear of the harvester to become fertilizer. These cameras check that no grains are being ejected – with the aim being to ensure zero wastage.

In just 30 years’ time, it is forecasted that the human population of our planet will be close to 10 billion.

Producing enough food to feed these hungry mouths will be a challenge, and demographic trends such as urbanization, particularly in developing countries, will only add to that.
To meet that challenge, agricultural businesses are pinning their hopes on technology, and that idea that increasingly sophisticated data and analytics tools will help to drive efficiencies and cut waste in agriculture and food production.
Leading the way is John Deere – the 180-year-old manufacturer of farming and industrial machinery which has spent the past decade transforming itself into an artificial intelligence (AI) and data-driven business.
Near the start of the journey in 2013, it unveiled its Farm Forward vision – demonstrating the concept of the “autonomous farm” where machinery would be remotely managed from a central control hub. It showed a farmer monitoring data points and managing machinery from a console in his home in real-time, while AI takes care of the moment-to-moment operational decisions.
Now it has released what it calls the 2.0 version of that vision – representing the leaps in learning and practical application of smart, self-teaching technology that has been made since those early days of the digital transformation.
I spoke to John Stone, SVP of John Deere’s Intelligent Solutions Group (ISG), about what has been learned since it starting out on this journey, and how it is pushing towards its latest vision of the future of agriculture.
Stone tells me "Now we're very much pushing into artificial intelligence, computer vision, and machine learning … what's amazing is how all of this technology is such a hand-in-glove fit for agriculture.
“It’s very clear that we need to be on the vanguard of these technologies – there’s a lot of economic upside and profitability as well as sustainability that can be unlocked for farmers through them.”
How easy is it to sell farmers on this vision of the future, though? After all, the common stereotypical portrayal of a farmer isn’t always a person on the cutting-edge of technology.
“It may not be the stereotype,” Stone tells me, “but farmers are an amazingly tech-savvy bunch.
“All of us know these new technologies are quite addictive and once they use them, they really like them.

“When we tell them they can spray their fields with 80 – 90% less herbicide, based on Blue River's testing … that's real money right in your pocket. As well as less herbicide going onto the plants that are going to become our food. Farmers are business people, and they're looking for business outcomes from this precision agricultural technology."
Blue River is the Silicon Valley-based machine learning startup acquired by Deere back in 2017, which has provided the foundation of the multinational giant's foray into artificial intelligence, with their expertise in computer vision, in particular, proving itself an incredibly valuable asset.
Computer vision is essentially the science of teaching computers to "see"– interpreting images captured by cameras to understand what they show and enable autonomous decision-making based on what is learned.
Stone says “The farmer has been the primary ‘sensor’ on a farm for years – and so much of farming is visual.
“It’s how does the ground look, what can you tell about the health of a plant by how it looks? Are the leaves nice and lush or are they going yellow? Are there bugs?
“The revolution taking place with deep learning has opened doors to solving problems that farmers have dreamed about solving for years … with computer vision systems and deep neural nets, there’s a very exciting future in these technologies in farms.”
One application of Blue River’s technology has been in the development of Deere’s See and Spray pesticide and herbicide distribution systems. This involves using smart cameras powered by computer vision, which are able to distinguish between healthy and unhealthy crops as machinery passes through the field. While traditionally the decision about whether or not to dose a crop with chemicals has been made on a field-by-field basis, this system allows targeted bursts of chemicals to be directed precisely where they are needed, at individual plants – hence the 80 to 90% reduction in herbicide use touted above.
Stone moved into his role with the ISG in 2016, just before the Blue River acquisition, meaning he has overseen the partnership between the industrial giant and the AI upstart from the outset.
“If you look at how many big, old industrials have successfully acquired and joined forces with a Silicon Valley start-up, I’d guess it’s not that many – we’re off to a fantastic start.”
As evidence of how well the two companies have merged cultures and learned from each other, he says of the 90 Blue River staff whose roles were acquired, just one has resigned since the takeover.
“The culture of the team is brilliant – some of the best deep learning and machine learning scientists there are … and full-stack software programmers … who just happen to have this tremendous affinity for agriculture. They want to be practicing their technology in an application that matters – not just in research papers.”
While the See and Spray system is still not quite ready for commercial deployment, John Deere has other machine and deep learning systems which are already in the hands of thousands of farmers across the globe.
One of these is the Combine Advisor system. Again, built around computer vision, this involves using cameras mounted on combine harvesters that monitor video images of grains as they are taken up the combine’s elevator and into the tank.
Deep neural networks are used to analyze the quality of the grain and make adjustments to the
operating parameters of the machinery on-the-fly if grains are getting damaged. More cameras
monitor the detritus from the harvesting operation – stalks, leaves, and cobs – as they are ejected from the rear of the harvester to become fertilizer for the fields. These cameras check that no grains are being ejected – with the aim being to ensure zero wastage.
Another crucial part of Deere's strategy to remain on the vanguard of AI and agriculture is its JD Labs “start-up collaborator." This enables it to be in close proximity to small, agile start-ups that are able to develop and trial innovative technologies in ways that might not be possible for a giant of John Deere’s size.
"It's a really cool program, that helps us to continue to stay engaged with a really exciting community," Stone says. "We provide some mentoring and agricultural and agronomic domain expertise, while they get a chance to continue to work on their technology and refine it, and we see what comes out at the end."