Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty

Hamed Farkhari; Joseanne Viana; Pedro Sebastiao; Luis Bernardo; Sarang Kahvazadeh; Rui Dinis

Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty

Hamed Farkhari
, Joseanne Viana
, Pedro Sebastiao
, Luis Bernardo
, Sarang Kahvazadeh
, Rui Dinis

Research output: Contribution to journal › Article › peer-review

Abstract

This research highlights the negative impact of ignoring uncertainty on DNN decision-making and Reliability. Proposed combined preprocessing and post-processing methods enhance DNN accuracy and Reliability in time-series binary classification for 5G UAV security dataset, employing ML algorithms and confidence values. Several metrics are used to evaluate the proposed hybrid algorithms. The study emphasizes the XGB classifier’s unreliability and suggests the proposed methods’ potential superiority over the DNN softmax layer. Furthermore, improved uncertainty calibration based on the Reliability Score metric minimizes the difference between Mean Confidence and Accuracy, enhancing accuracy and Reliability.

Original language	English
Journal	CEUR Workshop Proceedings
Volume	3601
Publication status	Published - 2023
Externally published	Yes
Event	Workshop on Research Projects Track at 17th International Conference on Research Challenges in Information, RCIS 2023 - Corfu, Greece Duration: 23 May 2023 → …

Keywords

5G
6G
Calibration
Deep Neural Networks
Jamming Identification
Reliability
Uncertainty
Unmanned Aerial Vehicle

Cite this

@article{b589e9215fa94d9abd2034b7ba52c13a,

title = "Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty",

abstract = "This research highlights the negative impact of ignoring uncertainty on DNN decision-making and Reliability. Proposed combined preprocessing and post-processing methods enhance DNN accuracy and Reliability in time-series binary classification for 5G UAV security dataset, employing ML algorithms and confidence values. Several metrics are used to evaluate the proposed hybrid algorithms. The study emphasizes the XGB classifier{\textquoteright}s unreliability and suggests the proposed methods{\textquoteright} potential superiority over the DNN softmax layer. Furthermore, improved uncertainty calibration based on the Reliability Score metric minimizes the difference between Mean Confidence and Accuracy, enhancing accuracy and Reliability.",

keywords = "5G, 6G, Calibration, Deep Neural Networks, Jamming Identification, Reliability, Uncertainty, Unmanned Aerial Vehicle",

author = "Hamed Farkhari and Joseanne Viana and Pedro Sebastiao and Luis Bernardo and Sarang Kahvazadeh and Rui Dinis",

note = "Publisher Copyright: {\textcopyright} 2023 CEUR-WS. All rights reserved.; Workshop on Research Projects Track at 17th International Conference on Research Challenges in Information, RCIS 2023 ; Conference date: 23-05-2023",

year = "2023",

language = "English",

volume = "3601",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

publisher = "CEUR-WS",

}

TY - JOUR

T1 - Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty

AU - Farkhari, Hamed

AU - Viana, Joseanne

AU - Sebastiao, Pedro

AU - Bernardo, Luis

AU - Kahvazadeh, Sarang

AU - Dinis, Rui

PY - 2023

Y1 - 2023

N2 - This research highlights the negative impact of ignoring uncertainty on DNN decision-making and Reliability. Proposed combined preprocessing and post-processing methods enhance DNN accuracy and Reliability in time-series binary classification for 5G UAV security dataset, employing ML algorithms and confidence values. Several metrics are used to evaluate the proposed hybrid algorithms. The study emphasizes the XGB classifier’s unreliability and suggests the proposed methods’ potential superiority over the DNN softmax layer. Furthermore, improved uncertainty calibration based on the Reliability Score metric minimizes the difference between Mean Confidence and Accuracy, enhancing accuracy and Reliability.

AB - This research highlights the negative impact of ignoring uncertainty on DNN decision-making and Reliability. Proposed combined preprocessing and post-processing methods enhance DNN accuracy and Reliability in time-series binary classification for 5G UAV security dataset, employing ML algorithms and confidence values. Several metrics are used to evaluate the proposed hybrid algorithms. The study emphasizes the XGB classifier’s unreliability and suggests the proposed methods’ potential superiority over the DNN softmax layer. Furthermore, improved uncertainty calibration based on the Reliability Score metric minimizes the difference between Mean Confidence and Accuracy, enhancing accuracy and Reliability.

KW - 5G

KW - 6G

KW - Calibration

KW - Deep Neural Networks

KW - Jamming Identification

KW - Reliability

KW - Uncertainty

KW - Unmanned Aerial Vehicle

UR - https://www.scopus.com/pages/publications/85182023112

M3 - Article

AN - SCOPUS:85182023112

SN - 1613-0073

VL - 3601

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

T2 - Workshop on Research Projects Track at 17th International Conference on Research Challenges in Information, RCIS 2023

Y2 - 23 May 2023

ER -

Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty

Abstract

Keywords

Other files and links

Fingerprint

Cite this