In this article:
How SONAH parking sensors achieve a detection accuracy of over 98% and how it is determined
In-depth insights into optimizing our detection accuracy / Expert interview
How we address issues with detection accuracy of our parking sensors
How do we at SONAH ensure that our camera-based parking sensors consistently operate with the highest accuracy? In this blog post, our Lead Software Developer Tarek takes you behind the scenes and shows how we determine and continuously improve the detection accuracy of our parking sensors.
SONAH's camera-based parking sensors (overhead sensors) are already playing an active role in rethinking parking management. However, no technology, no matter how advanced, can guarantee absolute detection accuracy. That's why we place special emphasis on analyzing and eliminating detection inaccuracies. This allows us to continuously improve the efficiency and reliability of our parking sensors.
What do we mean by detection accuracy?
Detection accuracy describes the precision with which a system—like our camera-based parking sensors (overhead sensors)—detects objects or events. In SONAH's case, this refers to how accurately our parking sensors identify vehicles in parking spaces. For example, if there are 1,000 parking spots and our system correctly determines the occupancy status (free or occupied) for 987 of them, the detection accuracy would be 98.7%.
How does SONAH ensure the high detection accuracy of its parking sensors?
"Don't underestimate the value of preparation"—a core company value at SONAH and the first step toward achieving the highest precision. Even before we install the first sensor, we lay the groundwork for top-notch detection accuracy with a precise project conception.
The project begins with a thorough analysis of the location. This involves either obtaining a comprehensive documentation (photos & videos) of all parking spots from the client or conducting an on-site visit ourselves to capture the conditions directly and.
Based on the collected parameters, we determine how many parking sensors are needed, where they should be installed, and at what angle to cover all parking spots. We ensure that each spot is optimally covered by the sensors and minimize coverage issues caused by other vehicles.
If the installation takes place in winter or spring, we also need to check whether potential coverage by trees could occur in summer and, if necessary, adjust the position or plan for pruning. Only when the parking sensor has an unobstructed view of the parking spots can the occupancy status be reliably determined.
In all these steps, we draw on over 8 years of experience in the field of smart parking. This extensive experience allows us to carry out the planning of our parking sensors based on proven best practices that have developed from our long-term work in the industry.
Once the sensors are carefully installed at the correct height and angle, the optimization of detection accuracy moves to the software side. Initially, configuration is crucial to ensure that the sensor is correctly adjusted to the conditions. For example, the sensor's exposure settings are established, which is particularly important in locations with low light at night. While exposure adjustment occurs automatically during operation, the initial configuration involves setting site-specific algorithms tailored to the conditions (more on sensor exposure can be found in the interview with Tarek below). Additionally, the "Regions of Interest" are precisely defined—these are the areas where the sensor checks for the presence of a vehicle, such as parking spot markings. Optimal definition of these regions significantly impacts the final detection accuracy.
Another key component of optimization is training neural networks with large datasets. Depending on the local conditions, we use situation-specific neural networks (Iterative Domain Specific Classifier based on a hypertuned dataset from 8 years of project work). Unlike a general classifier*, our neural networks are trained and calibrated for the specific conditions on-site right from deployment. This means the system is adjusted to handle factors such as heavy shading from trees or particular lighting conditions. This tailored fine-tuning enables us to maintain high detection accuracy even under challenging conditions.
*What is a general classifier?
A general classifier is a neural network or classification system designed for a wide range of tasks. It is intended to make general classifications without being specifically tuned to the conditions of a particular site or use case. While general classifiers are used in other areas, their detection accuracy is too low to be applied effectively in parking sensors.
The open-source library Keras, acquired and further developed by Google, offers a wide range of general classifiers, some of which are among the most accurate in their category. However, even the highest-accuracy classifier that can run on edge devices, "MobileNetV2," achieves only about 72% detection accuracy**—far too low for our use cases.
**Source: Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2019). MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv Preprint arXiv:1801.04381v4. Google Inc. Retrieved August 28, 2024, from https://arxiv.org/abs/1801.04381
In contrast to the systems of many other manufacturers, the plausibility check of our data is a central component of our quality management. We have implemented a data sanitizing algorithm that detects anomalies in the datasets. Our principle is: Only plausible data is sent out. If anomalies are detected, such as excessively frequent status changes within one second or similar issues, our system automatically switches the affected parking spot to a "No Data" mode—because no data is better than incorrect data.
We then review the affected parking spot, analyze possible causes for the anomalies, and coordinate with the customer to align the on-site situation with the data. If necessary, we adjust the parameters or take further actions to optimize detection accuracy and ensure precise results.
Different detection methods
Depending on the application of the sensors, the detection methods vary. In the case of single spot detection, there are fundamentally two different use cases:
Marked Parking Spots: Detection is binary—meaning the sensor identifies whether a vehicle is present or not within a predefined area (the region of interest).
Unmarked Parking Spots: Here, the region of interest covers the entire area of the unmarked parking spots. An algorithm not only detects vehicle occupancy but also locates detected objects (cars, vans, trucks, etc.) within this region of interest to determine how much and where parking space has been occupied. With an accuracy of +/- 10 cm, our parking sensors can measure a gap.
How do our customers measure detection accuracy?
To ensure the data quality of the systems, many of our customers conduct regular on-site inspections. During these checks, the parking area is manually surveyed to compare the data displayed on the frontend after a 30-second wait with the actual on-site situation.Die Wartezeit ist vorgesehen, um Situationen zu vermeiden, in denen gerade ein Parkvorgang stattgefunden hat, aber das Frontend noch nicht die Live-Daten anzeigt.
The wait time is designed to avoid situations where a parking event has just occurred but the frontend has not yet updated with the live data.
In case of discrepancies, these are carefully documented: a photo of the parking spot is taken, a screenshot of the frontend is captured, and the parking spot ID is noted. Detection accuracy is calculated by dividing the number of discrepancies by the total number of parking spots.
The accuracy of detection strongly depends on the sample size and the quality of the data input. It is important to ensure that at least 100 different measurement points are recorded and that both the times of day and weather conditions during the measurements are varied.
To gain deeper insights into the optimization of our detection accuracy, we interviewed our Lead Software Developer, Tarek. He explains what key elements are crucial for the high precision of our sensors and shares his practical expertise.
Which parameters are crucial for detection accuracy?
"Several factors play a role in calculating detection accuracy. These include the selection and training of the vehicle types the system is meant to recognize, as well as accounting for specific environmental influences. Particularly challenging are weather conditions such as heavy fog or snowfall, which can impair visibility. Additionally, we must consider the impact of trees or other obstacles that could obstruct the camera sensors."
How do the evaluation methods differ?
"The evaluation methods for different detection systems vary significantly in how occupied and free parking spots are detected. In individual parking spot detection, each spot is directly checked for its status. The result is binary: a parking spot is either 'occupied' or 'free'.
The detection is based on the agreed product specifications and typically includes vehicle types such as cars, vans, and pickups. Vehicles that do not fall into these categories must be disregarded during validation. The system is trained to ignore foreign objects or compact vehicles. These parameters, like nearly all parameters, can be customized to meet specific customer requirements.
In contrast, detection for unmarked parking spots is based on measuring the gaps between vehicles or between a vehicle and the end of the parking area. The system calculates how many vehicles of a predefined size (which can be adjusted) would fit into these gaps.
For example, a rule might assume a vehicle length of 5 meters and an additional buffer zone of 75 cm. Free spaces that are large enough to accommodate at least one more vehicle are displayed in the frontend with a number indicating the count of additional vehicles. Smaller free areas, which are not large enough to fit another vehicle, are shown but do not carry a number. These differences in detection and display allow us to provide both precise results for marked parking spots and a flexible occupancy overview for unmarked areas."
(Screenshot of the visualization of unmarked parking spots in our userinterface of our web dashboard)
What challenges are there in the statistical processing of data?
"In the statistical processing of data, there is always the risk of measurement inaccuracies. Such inaccuracies can be caused by various factors, such as unusual vehicle types or unexpected obstacles in the sensor's field of view.
However, the system is designed to minimize these factors. To ensure the accuracy and reliability of the system, it is crucial to conduct regular inspections. These checks help identify and correct inaccuracies early on. They are complemented by automated algorithms that dynamically adjust to environmental influences. An example of this is automatic exposure adjustment. This feature ensures that the sensor adjusts its parameters according to varying light conditions, whether in darkness or overexposure.
To support ongoing maintenance measures, we create light profiles for each installation of a camera-based parking sensor. These profiles ensure precise occupancy detection even under changing lighting conditions. With these customized exposure profiles, we achieve a detection accuracy of +98%, even at dusk or in darkness."
Lighting profiles from our project with the city of Wuppertal:
(Image: Screenshot, lighting profiles from the city of Wuppertal. You can see when the lanterns are switched on at the respective locations)
How do we address issues with detection accuracy?
With over 100 projects in more than five countries, SONAH has gained extensive experience in the field of smart parking. In each project, we face the challenge of achieving a high detection accuracy of +98%. To ensure this, it is crucial to establish a common information base to determine whether an issue is genuinely related to detection accuracy or if other factors, such as frontend misinterpretations, defective parking sensors, backend failures, or incorrectly implemented API queries, are at play.
Let's consider the following scenario:
“A customer reports a low detection accuracy that they have measured over an extended period, according to the process we provided. The sample consists of approximately 20 measurements, including brief descriptions of the detection errors and images of the actual situation of the errors.”
Step 1 - Analysis & Data Consolidation
Customers can use an online form provided by us for easy submission of samples. The evaluation is conducted by our project management team in collaboration with our software team. We cross-reference the information provided by the customer with the dedicated problem areas and the digital parking data. This allows us to identify which parking sensor is causing issues and determine what adjustments are needed.
Step 2 - Deriving Actions
Based on the images provided by the customer and the specific neural network of the parking sensor, we adjust the system. A software developer reviews the input according to the error profile submitted by the customer and makes corresponding adjustments to the network. Additionally, we can supply the neural network with "problem-specific" annotated training material, such as "...frequent detection errors caused by parked motorcycles in a parking spot..."
This allows the neural network to learn how to handle the object, reducing the error rate of the parking sensor. The training material is generated from research projects where we act as project partners. Similar approaches are taken for complex weather conditions or shadows.
The now improved neural network is uploaded to the parking sensor over-the-air. This allows us to perform parking spot-specific detection optimizations through software updates. Such individual adjustments are only achievable with camera-based parking sensors.
If an error was made during the project planning or installation, if structural conditions have changed, or if complex shading/obstruction scenarios occur, there is the possibility to physically realign the existing sensors or deploy additional technology. In some cases, the problem cannot be fixed via a software update. However, this is an exception.
(Image: Selection of clients, who confirmed our detection accuracy)
Step 3 - Obstruction & Additional Sensors
If regular obstructions by large objects occur, we recommend installing additional parking sensors to cover all parking spots digitally in as many scenarios as possible and ensure the detection accuracy of the parking sensors. In cases where an additional camera-based parking sensor is not feasible, we offer ground sensors to ensure the digitization of parking space management.
This systematic approach enables cities and municipalities to effectively optimize their parking management strategies and meet the demands of modern mobility solutions.
The precision of our parking sensors is the result of continuous effort and detailed processes that we at SONAH implement with passion and expertise. From the initial project conception to the fine-tuning of our software and the regular review of data quality—our goal is to deliver reliable and accurate results.
We hope this insight into our approach and the challenges in ensuring high detection accuracy provides you with valuable perspectives.
Thank you for your interest, and stay tuned for upcoming updates and innovations from SONAH! If you want to learn more about our processes or start a project with us, please don’t hesitate to contact us.
