Unslotted csmaca optimization method and devices in wi-sun using reinforcement learning

Abstract

An unslotted CSMA/CA optimization device includes a variable initializing unit performing variable initialization used in an algorithm for unslotted CSMA/CA optimization; an exploration and exploitation selecting unit determining exploration/exploitation using an epsilon greedy algorithm; an action selecting unit selecting an action having the best Q-value, among actions, when exploitation is selected, and randomly selecting an action when exploration is selected; a channel information collecting unit executing backoff when an action (backoff time) is selected, repeatedly executing CCA during the backoff time, and counting the number of times a channel is idle and the number of times the channel is busy; a success rewarding unit transmitting a packet when the channel is idle and rewarding success when acknowledge (Ack) is received; and a Q-table updating unit checking the received reward and updating a Q-table based on an action, a state, and a reward.