The development of IoT (Internet of Things) systems is tending towards greater complexity, as the number of connected objects in the IoT sector is increasing at an impressive rate, whether in healthcare, public transport or industry 4.0... The architectures and algorithms developed today to drive these systems are based on the use of innovative technologies such as artificial intelligence. However, the current paradigm remains the development of human-powered intelligence to solve problems that, while complex, do not evolve. Yet in new contexts, such as smart cities, human intervention can become a bottleneck or even a point of failure, and the environment is constantly changing, necessitating the adaptation of constantly-developed algorithms.

To ensure that new IoT systems function properly, a paradigm shift from intelligent to autonomous IoT systems [1] is essential. Tomorrow's IoT systems will only exist if they are autonomous, driven by algorithms that require no human intervention and are capable of adapting to any changes in their environment autonomously. To achieve this autonomy, the algorithms we are developing are based on the technique of deep reinforcement learning (DRL), capable of automatically adapting to new contexts. Thus, an agent continuously interacts with the environment, using observations about the environment to take actions that the environment should implement. The right actions reinforce the agent's learning through rewards from the environment. Unlike conventional artificial intelligence techniques, the agent doesn't need any data prepared by a supervisor (a human); it only uses its own experiences to improve its knowledge of the right actions to take. More importantly, the agent doesn't solve a fixed problem that doesn't evolve. The agent's learning process is perpetual, enabling its algorithm to adapt to new configurations and continue to improve its performance.

Several challenges remain to be overcome in order to achieve autonomous IoT systems, two in particular:

• Firstly, conventional DRL architectures are mostly centralized, due to the agent's high computational requirements. Such architectures are unreliable in a real-world environment. Architectures need to be distributed as close as possible to the action zones.
• Secondly, developing an intelligent agent from scratch each time is likely to pose serious scaling-up problems. It must be possible to transfer intelligence from one domain to another.

The work we are planning aims to address these challenges in order to realize autonomous IoT systems. Preliminary results obtained in the field of task offloading [2] and QoS management in rural areas [3] point the way to this realization.

[1] L. Lei, Y. Tan, K. Zheng, S. Liu, K. Zhang and X. Shen, "Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges," in IEEE Communications Surveys & Tutorials, vol. 22, no. 3, pp. 1722-1760, thirdquarter 2020, doi: 10.1109/COMST.2020.2988367.

[2] Abdel Kader Chabi Sika Boni, Hassan Hassan, Khalil Drira, « Task Offloading in Autonomous IoT Systems using Deep Reinforcement Learning and ns3-gym » , 11th International Conference on the Internet of Things (IoT 2021), Nov 2021, St Gallen, Switzerland

[3] Juan Chafla Altamirano, Mohamd Amine Slimane, Hassan Hassan and Khalil Drira, « QoS-aware Network Self-management Architecture based on DRL and SDN for remote areas », submitted