Video Cluster Counting

Let’s say you have a row of grapes that represents average yields in your entire vineyard. By counting clusters in that row you have a quick measure of the relative cluster counts over that row’s length. This measure can then be easily expanded across the entire vineyard area.

To help with this process, we have developed some open resources that can use a cellphone video as input, and a cluster count as output. We have written a public Google Colaboratory (Colab) that is freely available and can detect, track, and count flower clusters in video files. This makes it possible to get a rough count of clusters over a given distance in a vineyard. Here we’ll walk you through the steps to use our Colab notebook—a free tool from Google—to upload and analyze your own cluster videos using one of our pre-trained models. This Colab notebook is compatible with custom object detection models as well.

Getting Started

To use our video cluster counter, you will need a few things:

• A Google Account

  • Colaboratory runs on Google Drive

• A video file

  • This is the actual video of your vines.

Capturing Video

In this tutorial we will be using ‘nighttime-grape-flower-clusters/1’ which is a nighttime cluster counting model. You can explore our other cluster counting models or train your own model.

Note: ‘nighttime-grape-flower-clusters/1’ is a nighttime-only model. This means you’ll need to capture your video at night using a lighting source. We recommend placing light sources a few feet above and/or below your camera as in the figure below:

Open

Jaramillo et al., 2021

You will want to experiment with different frame rates and video lengths to see how effectively they work with our cluster counter. The video used in this tutorial was filmed at 250 fps and is 17 seconds long. We are actively testing much lower frame rates (60 fps) and longer videos and we will update this document as we come up with further recommendations. Here is the video we’ll be using in this example:

Get Organized

1. Sign in to drive.google.com using your Google account. Click on ‘My Drive’ to see any existing files/folders in your Google Drive. Click ‘New’ → ‘New folder’ to create a new folder for your project. For the sake of this tutorial, call the folder ‘Computer_Vision’.

   Open

   

2. Next, double-click into your ‘Computer_Vision’ folder. Inside this folder, we will upload our model weights alongside the video file you have captured with your phone. For this tutorial, download our ‘nighttime-grape-flower-clusters’ model and upload it to your ‘Computer_Vision’ folder. You can download the file here:

3. At this point, you should have a video file and a trained model next to each other in your ‘Computer_Vision’ folder on Google Drive. In this tutorial, we’ll name our file ‘test.MOV’. Don’t worry if your video file is in a different format; our processing script will convert it later.

Configure the Colab

1. In a new tab, open up our Colab notebook: https://colab.research.google.com/drive/1zMc-4Bjtxt-Ye2N4p2VKG3wqWovmNzmG?usp=sharing

2. Click ‘Copy to Drive’. This will copy the notebook to your Google Drive and open it in a new tab.

3. In your copy of the Colab, go to the ‘Runtime’ menu and select ‘Change runtime type’. Make sure a GPU option is selected such as 'T4 GPU'.

4. Finally, adjust the settings block. Here are what each variable in that block means:

   • RAW_VIDEO_PATH - This points to your Google Drive (/content/drive/MyDrive/) and the location and name of the raw video you uploaded.

   • CONVERTED_VIDEO_PATH - You do not need to change this. This is where the script will copy your video to after converting it to a .mp4

   • MODEL_PATH - This points to your Google Drive (/content/drive/MyDrive/) and the location and name of the model you uploaded.

   • FINAL_VIDEO_PATH - You do not need to adjust this. This tells the script where to save the final video with clusters counted.

   • SAVE_PATH - This points to your Google Drive (/content/drive/MyDrive/) and the location and name of the final video you want saved back to your drive.

Run Through the Script

At this point, you should be all set to analyze your video and count the clusters! Simply step through one code block at a time pressing the little play button on the left and waiting for a checkbox to appear before moving on to the next block. Some blocks will take a significantly longer time to run. Be sure not to close the Colab or you will have to start at the top again (despite the checkboxes remaining).

Finished!

When all of the code blocks have been run, you should have a processed video on your Google Drive. From there, you can download the video or stream it directly. The final result will look something like this:

Next Steps