Hi everyone,
Here I share the first results from the comparative test we carried out to see differences in terms of performance on bats between recording devices and membranes. The tested devices were:
1- Audiomoth v1.1.0 (old version)
2- Audiomoth v1.2.0 (new version)
3- SM4 + U1 microphone (x2)
4- SM4 + U2 microphone
Furthermore, each version of Audiomoth were set with 3 different membranes plus a fourth one with no casing:
1- Akustikstoff: https://www.akustikstoff.com/Buy-Cuts-of-Speaker-Cloth-in-Various-Sizes/Acoustic-Cloth-2-0-Pre-cut-50-x-160-cm-19-7-x-61-42-Colours::764.html
2- Hydrophobous fabric (TNT), similar to the light fabric used in dinning tables: (in Spanish) http://www.tejidonotejido.com/es/caracteristicas-tejido-no-tejido/
3- Metal sticker (PP): (in Spanish) https://www.apli.com/es/producto/10066
The devices were placed in a window facing an urban area, at the same angle an elevation. All AM with membranes were using the same type of case. All devices recorded for 4h from 1h after sunset on the 16th of February 2021.
The new AM v1.2.0 detected less batpasses (fragments of 5s with at least 1 bat call) and calls (individual pulses) in all four membrane configurations, with different results depending on the membrane used.
Differences in batpasses ranged between 8% less (no case) and 43% less (PP membrane). When looking at species level, these values changed, only showing similar results in the commonest species the Kuhl's Pipistrelle. Much scarcer species like Common and Soprano Pipistrelle showed different results, showing much better performance on the former one compared to old AM v1.1.0 in all membrane configurations.
Regarding membranes, the Akoustikstoff and TNT showed similar results, having 18-21% less batpasses with the new AM v1.2.0 than the no casing configuration. The PP showed very poor performance. Surprisingly, comparing the AM with SM4, both showed similar results in terms of batpasses, which is the usual variable used in bat ecology studies.
I'm waiting for more detailed analysis from other tests done with the same devices, but the results so far point towards the following conclusions:
1- The new AM version has a worse performance in medium freq. bats (pipistrelles) than the old AM version. I think having worse numbers in all 4 membrane configurations is consistent enough to conclude this. The different results when looking at scarcer species might be due to low sample size.
2- The TNT membrane offers a good alternative to the Akoustikstoff membrane.
3- The AM may not be as bad as it seems compared to SM4 when analysing batpasses, at least for medium freq bats.
Has anyone done any similar testing, especially regarding old vs new AM version? Or could anyone explain the difference in performance between species?
Cheers,
Robert
Hi Edwin, I tested one device for each membrane and version, so one replicate. But keeping in mind all four membrane types showed poorer results on the AM 1.2.0., I think the differences between AM versions are real.
I'll try to find some time and check the quality of the sonograms, but I don't think the noise would be the reason as all devices were recording simultaneously, so one noise would affect all devices.
Cheers,
Rob
How many devices did you test of each type? It kind of seems that people are reporting differences even between the same models. I think it is hard to tut any conclusions to testing just 1 or a few pcs per model. I do not own 1.1.0 and 1.2.0 versions but only a few of an adapted version 1.1.0 with SPU0410LR5H microphone. It seems this microphone has a bit more sensitivity on the higher frequencies. This could explain why a pigmaeus can have a better score than the kuhls but if I see correctly ppip is performing much better? I think it would be nice to look at the sonograms to try and undersand the difference. Maybe some noise is in the way?
Thanks Robert for doing the work and reporting it. I'm intrigued about the difference in performance between the membranes. Are these full cases with only a small (membrane covered) hole for the microphone, or is the whole front of the case missing?
Thanks
Wim
Very interesting, andya. We haven't considered using the amplitude threshold option yet, but we'll certainly keep your results in mind when we do it. Also, it's surprising to see results vary depending on sampling rate!
Important to say in the comparison done above all AM were set to the same configuration: 250kHz Sampling Rate, Medium Gain, No filters or threshold and sleep/recording duration of 0/3600s. SM4 were set to 256kHz, 0dB gain, and no trigger and file length of 3600s. So the observed differences are not due to different configurations.
Cheers,
Robert
Hi Robert, great work!
The summary of my testing results are published here: https://somersetbat.group/advice/which-bat-detector/audiomoth/testing/
I was focusing on establishing the best AM config settings for UK bats, and the short summary is "the selected configuration has a huge impact on the number of bat passes detected per species" So I think it is key to factor AM configuation into your results when AM testing :-)