Close
0%
0%

Ai Honey Bee Swarm Detector / Predictor System

Local C02 sensor and weather data scraped off interwebs predict swarming

Similar projects worth following

Swarming is an important part of the honey bee's reproductive process and one colony can send off many swarms when it's hive has become over crowded or otherwise undesirable. There are many species of bees and this project focuses on the native honey bees in my location, Apis, Mellifera Mellifera.  Sadly, these bees are under significant threat from other honey bees such as Italian or Buckfast that are imported by other bee keepers in the belief that they are more productive. However, the local bees have many advantages, one being that they have developed behavioral techniques to combat the pervasive Varroa mite which then means that the hives do not need to be treated with aggressive chemicals. This project aims to produce a fairly simple system to predict the arrival of incoming swarms so that the bee keeper can then go and physically look for the swarms and give them a home. These boxes could be put in a multitude of different locations as long as they are in range of a LoRa gateway and can not only predict the arrival of a swarm but also tell the beekeeper if a swarm has taken up permanent residence in the catch hive.

Bees respire - they breathe in oxygen and breathe out CO2 and this can be measured and used to estimate bee numbers inside a catch hive.

A catch hive is a special box positioned high above the apiary in a location where bees find it attractive to scout for their next home. Scouting activity is often indicative of a swarm being formed somewhere in a 3 mile radius of the box and often eventually results in the arrival of a swarm of at least 1,000 bees. To witness a swarm of bees is one of the most amazing experiences. First, you hear a strange buzzing noise over head which can be quite loud. Next, the sky darkens and a huge cloud of black insects comes slowly towards you. Fortunately, they are normally completely non-aggressive and tend to stay at well above human head level.

Last year, I counted 13 swarms arriving in my own apiary, which is not unusual. It's good to try and catch the swarms as they are valuable to the beekeeper.

PCBs for this project were kindly supplied free by Seeed Studio.

Work tasks:

  1. Rig LoRa darkStar up to a honey bee catch hive with external pipes to measure CO2 ..... DONE.
  2. Build online database or CSV file to hold values ....... DONE.
  3. Build script to rip local weather stats.from the interwebs ...... DONE.
  4. Collect data - the swarming season is imminent!  ........ IN PROGRESS.
  5. Add CO2 value, swarm boolean and weather details at each time point to database / CSV file. ....... IN PROGRESS.
  6. Design a machine learning system to train and infer swarming. Will probably use a XGboost classifier, which has worked well in the past. ....... DONE.
In past years, I've noticed that the bees dont just randomly form swarms, but will wait until the weather conditions are amenable. Normally, they will swarm on a warm, humid and sunny afternoon and not when it's windy or raining.. The swarming instinct is partially triggered by over-crowding in their original home but will also be highly dependent on available food resources and average temperatures as the queen is quite happily able to regulate the number of eggs she lays. It's not possible to collect data about food resources, but average temperatures is no problem. Apart from weather factors, swarming in my location also occurs at the start of May, so adding data about the time of year is going to be useful. Finally, I will need to physically walk down to the apiary every day and look for swarms and maybe inspect the catch hive if it has high CO2 levels when it is dark, which would indicate that a swarm has probably taken up permanent residence.  These human observations are manually entered into the csv file as a Boolean ie either 0 or 1, no swarms visible and swarm visible.

The native honey bees, in my region Apis Mellifera Mellifera , are fascinating animals and are one of the few species able to operate true democracies which...

Read more »

pcb - 513.50 kB - 06/06/2023 at 08:00

Download

Zip Archive - 168.64 kB - 06/06/2023 at 07:59

Download

x-arduino - 20.24 kB - 06/06/2023 at 07:58

Download

  • Initial data analysis

    Capt. Flatus O'Flaherty ☠05/31/2023 at 19:14 0 comments

    I've chopped off a lot of the graph to the left as nothing much was happening, but more recently 1. A swarm appeared in the apiary but did not all go into the catch hive, shown encircled in red. 2. A swarm was caught inside the catch hive, circled by green.

    In both cases, there seems to be evidence of scouting behaviour prior to the swarm arriving and this was confirmed by human eyeball observations. There's a couple of strong peaks for CO2 in the red circle and one strong peak in the green zone. It would be great if another swarm arrived as this would re-enforce the current patterns (or maybe not!)

  • The newly caught swarm begins it's slow journey down the ladder

    Capt. Flatus O'Flaherty ☠05/28/2023 at 11:02 0 comments

    This very short video shows why it's important to move the new green box slowly, one step at a time. After moving it about 1/2 meter downwards, quite a few of the foraging bees have started to try and get back into the original white box. Fortunately, I knew they would probably try to do this and had the door closed. When all the bees have got accustomed to the new green box position, it can be moved down another couple of ladder rungs.

  • Juggling beehives at the top of a ladder

    Capt. Flatus O'Flaherty ☠05/27/2023 at 14:37 0 comments

    The white box with the LoRa Darkstar attached was opened up and was found to contain a large swarm of bees, as expected. The bees needed to be transferred into another box, the green one, so that they can be removed from the top of the tree back down to ground level, leaving the white box in the tree to catch or detect another swarm. Howver, it's not possible to do the whole operation in one stage as the bees need time to re-orientate. The proceedure is as follows:

    1. Put the green box in the original position of the white box with the door open.
    2. Transfer all the bees from the white box to the green one by moving the frames and shaking out any stragglers.
    3. Close the door on the white box and position on top of the green one.
    4. Gradually move the green box down the ladder - about 1 foot per day.
    5. Put some old frames back in the white box and open the door again.

  • CO2 levels have rocketed

    Capt. Flatus O'Flaherty ☠05/27/2023 at 08:50 0 comments

    There must be a swarm in the box! Ambient CO2 levels are about 420 ppm.

  • Swarm caught

    Capt. Flatus O'Flaherty ☠05/26/2023 at 16:12 0 comments

    CO2 levels have shot up to over 2,000 ppm and it looks like a swarm has colonised the catch hive!

  • Lots of scouting activity but no swarm arrival ... yet

    Capt. Flatus O'Flaherty ☠05/26/2023 at 09:49 0 comments

  • Intense scouting coupled with elevated CO2 levels

    Capt. Flatus O'Flaherty ☠05/25/2023 at 18:43 0 comments

    CO2 levels have just peaked at 855 ppm (ambient level is 420) which indicates intense scouting or even a swarm taking up permanent residence. If the level stays high during the night, this could indicate permanent occupation. Otherwise, it could take a day or two for the actual swarm to arrive.

  • Catch hive high up in hawthorn tree

    Capt. Flatus O'Flaherty ☠05/19/2023 at 14:32 0 comments

    If you look carefully inside the red disc near the entry hole there is a scout bee, probably from another hive starting to go into swarm mode. However, one single bee is not likely to affect the inside CO2 levels very much!

  • First swarm of the season seen in apiary

    Capt. Flatus O'Flaherty ☠05/17/2023 at 13:04 0 comments

    It would have been nice to put up a photo of the actual swarm, but it was all a bit frantic trying to catch it before it absconded to under the neighbour's  facia board or into the inside of a post box. But the data is equally of interest:

    The underlying theory of the whole thing is that measuring CO2 levels in the catch hive can help indicate and ultimately predict swarming. There's a whole load of other factors which are recorded, but CO2 was thought to be the most important. One of the great things about XGBoost is that it can automatically tell us which factor is ACTUALLY most important with a nice graphical display, but this will all come later when we have a bit more data to work with. One swarm aint enough.

    Fortunately, I set the system up to calculate the max CO2 value in the last 48 hours as I knew that scouting bees would be active before the swarm actually materialized. What is really strange is that the scouting bees seem to have decided to spend the night in the catch hive, no doubt to check out the furnishings and raid the mini bar. After the swarm has now been collected and re-housed the CO2 patterns seem to have returned to normal.

    Another indicator that I thought might be important is the gadget temperature since it already has a temperature sensor in it and is in full sunlight, just like a bee hive. It gets hot, but bees like this degree of heat and are more than capable of ventilating the hive when necessary.

  • XGboost Code Tested

    Capt. Flatus O'Flaherty ☠04/29/2023 at 14:36 0 comments

    I found some nice code for testing XGboost classification. It was a bit out of date, but with some judicious tweaking I managed to bring it up to date and work nicely.

    #!/usr/bin/env python
    # cd /home/flatus/Documents/solar_harvester/swarm_predicter && python3 xgboost_test_07.py
    
    # Data:
    # https://www.kaggle.com/datasets/liujiaqi/hr-comma-sepcsv?select=HR_comma_sep.csv
    
    import warnings
    warnings.filterwarnings("ignore")
    warnings.filterwarnings( "ignore", module = "matplotlib\..*" )
    warnings.filterwarnings( "ignore", module = "xgboost\..*" )
    
    import pandas as pd
    import numpy as np
    import xgboost as xgb
    from sklearn.model_selection import train_test_split
    import matplotlib.pyplot as plt
    from sklearn.preprocessing import LabelEncoder
    from sklearn.metrics import confusion_matrix
    from termcolor import colored
    
    print(colored("Start", 'blue'))
    # print(colored('The Accuracy on Test Set is: ', 'blue')
    
    df = pd.read_csv('/home/flatus/Documents/solar_harvester/swarm_predicter/HR_comma_sep.csv')
    print(df.head())
    
    #data process
    X = df[['satisfaction_level', 'last_evaluation', 'number_project', 'average_montly_hours', 'time_spend_company', 'Work_accident', 'promotion_last_5years', 'sales', 'salary']]
    y = df['left']
    
    # Extract text features
    cats = X.select_dtypes(exclude=np.number).columns.tolist()
    
    # Convert to Pandas category
    for col in cats:
       X[col] = X[col].astype('category')
    
    # Splitting the dataset into the Training set and Validation set
    Xt, Xv, yt, yv = train_test_split(X, y, test_size = 0.25, random_state = 0)
    # dt = xgb.DMatrix(Xt.as_matrix(),label=yt.as_matrix())
    # dv = xgb.DMatrix(Xv.as_matrix(),label=yv.as_matrix())
    
    dt = xgb.DMatrix(Xt, yt, enable_categorical=True)
    dv = xgb.DMatrix(Xv, yv, enable_categorical=True)
    
    
    #Build the model
    params = {
        "eta": 0.2,
        "max_depth": 4,
        "objective": "binary:logistic",
        "silent": 1,
        "base_score": np.mean(yt),
        'n_estimators': 1000,
        "eval_metric": "logloss"
    }
    model = xgb.train(params, dt, 3000, [(dt, "train"),(dv, "valid")], verbose_eval=200)
    
    #Prediction on validation set
    y_pred = model.predict(dv)
    
    # Making the Confusion Matrix
    cm = confusion_matrix(yv, (y_pred>0.5))
    print(colored('The Confusion Matrix is: ', 'red'),'\n', cm)
    
    # Calculate the accuracy on test set
    predict_accuracy_on_test_set = (cm[0,0] + cm[1,1])/(cm[0,0] + cm[1,1]+cm[1,0] + cm[0,1])
    print(colored('The Accuracy on Test Set is: ', 'blue'), colored(predict_accuracy_on_test_set, 'blue'))
    
    
    # Based on the model we made, we can predict a employee whether he may leave or not after inputing his informaion:
    print("")
    print("Now make a single prediction: ")
    
    satisfaction_level = 0.38
    last_evaluation = 0.53
    number_project = 2
    average_montly_hours = 137
    time_spend_company = 3
    Work_accident = 0
    left = 1   # Dont actually need this one as this is what we're trying to predict.
    promotion_last_5years = 0
    sales = "sales"
    salary = "low"
    
    # Make prediction:
    
    # Use a dictionary type:
    dict_ = {'satisfaction_level': satisfaction_level, 'last_evaluation': last_evaluation, 'number_project': number_project, 'average_montly_hours': average_montly_hours, 'time_spend_company': time_spend_company, 'Work_accident': Work_accident, 'left': left, 'promotion_last_5years': promotion_last_5years, 'sales': sales, 'salary': salary}
    my_HR_test = pd.DataFrame([dict_])
                     
    # Get rid of the 'left' column:
    my_HR_test = my_HR_test.drop('left', axis=1)
     
    # Extract text features
    cats = my_HR_test.select_dtypes(exclude=np.number).columns.tolist()
    
    # Convert to Pandas category
    for col in cats:
       my_HR_test[col] = my_HR_test[col].astype('category')
    
    my_HR_test = xgb.DMatrix(my_HR_test, enable_categorical=True)
    
    # Make prediction
    new_prediction = model.predict(my_HR_test)
    
    print("new prediction: ",new_prediction)
    
    if(new_prediction > 0.5):
        print(colored("This employee will leave!", 'blue'))
    else:
        print(colored("This employee will not leave!", 'green'))
    

    The key to success was to bring the 'One Shot' prediction data into Pandas as a dictionary. Not seen this done before, but...

    Read more »

View all 12 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates