POBLICATIONS

Date
Title
2023
Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Deep Reinforcement Learning (DRL) has the potential to surpass human-level control in sequential decision-making problems. Evolution Strategies (ESs) have different characteristics than DRL, yet they are promoted as a scalable alternative. To get insights into their strengths and weaknesses, in this paper, we put the two approaches side by side. After presenting the fundamental concepts and algorithms for each of the two approaches, they are compared from the perspectives of scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the paper discusses hybrid algorithms, combining aspects of both DRL and ESs, and how they attempt to capitalize on the benefits of both techniques. Lastly, both approaches are compared based on the set of applications they support, showing their potential for tackling real-world problems. This paper aims to present an overview of how DRL and ESs can be used, either independently or in unison, to solve specific learning tasks. It is intended to guide researchers to select which method suits them best and provides a bird’s eye view of the overall literature in the field. Further, we also provide application scenarios and open challenges.
2023
Chirpy — AI-based Audio Localization and Communication For Swarm Robots
Introducing Chirpy, a hardware module designed for swarm robots that enables them to locate each other and communicate through audio. With the help of its deep learning module (AudioLocNet), Chirpy is capable of performing localization in challenging environments, such as those with non-line-of-sight and reverb. To support concurrent transmission, Chirpy uses orthogonal audio chirps and has an audio message frame design that balances localization accuracy and communication speed. As a result, a swarm of robots equipped with Chirpies can on-the-fly construct a path (or a potential field) to a location of interest without the need for a map, making them ideal for tasks such as search and rescue missions. Our experiments show that Chirpy can decode messages from four concurrent transmissions with a Bit Error Rate (BER) of at a distance of 250 cm, and it can communicate at Signal-to-Noise Ratios (SNRs) as low as -32 dB while maintaining ≈ 0 BER. Furthermore, AudioLocNet demonstrates high accuracy in classifying the location of a transmitter, even in adverse conditions such as non-line-of-sight and reverberant environments.
2022
Covy — An AI-powered Robot for Detection of Breaches in Social Distancing
We present Covy — a robotic platform that promotes social distancing during pandemics like COVID-19. Covy features a novel compound vision system that enables it to detect social distancing breaches up to 16m away. Covy navigates its surroundings autonomously using a hybrid navigation stack that combines Deep Reinforcement Learning (DRL)and a probabilistic localization method. We built the complete system and evaluated Covy’s performance through extensive sets of experiments both in simulated and realistic environments. Amongst others, our results show that the hybrid navigation stack is more robust compared to a pure DRL-based solution.
2021
Lightweight Audio Source Localization for Swarm Robots
The ability to localize a sound source is essential for safe integration of robots in our society. However, its high computational demand makes it challenging to equip small swarm robots with such capability. Therefore, this paper investigates potential means for developing a lightweight sound source localization method. It introduces the Cutting The Plane (CTP) algorithm which leverages recent advancements in embedded microphone technology to reduce the computation complexity of audio source localization. We analyzed its performance and compared it to a software-based approach using simulation. Finally, we performed tests on real hardware to show the feasibility of using our system for relative localization between swarm robots.
2020
Continuous Sensing on Intermittent Power
The main obstacles to achieve truly ubiquitous sensing are (i) the limitations of battery technology – batteries are short-lived, hazardous, bulky, and costly – and (ii) the unpredictability of ambient power. The latter causes sensors to operate intermittently, violating the availability requirements of many real-world applications. In this paper, we present the Coalesced Intermittent Sensor (CIS), an intermittently-powered “sensor” that senses continuously! Although a single node will frequently be off charging, a group of nodes can –in principle– sense 24/7 provided that their awake times are spread apart. As communication is too expensive, we rely on inherent component variations that induce small differences in power cycles. This basic assumption has been verified through measurements of different nodes and power sources. However, desynchronizing nodes is not enough. An important finding is that a CIS designed for certain (minimal) energy conditions will become synchronized when the available energy exceeds the design point. Nodes employing a sleep mode (to extend their availability) do wake up collectively at some event, process it, and return to charging as the remaining energy is typically too low to handle another event. This results in multiple responses (bad) and missing subsequent events (worse) due to the synchronized charging. To counter this undesired behavior we designed an algorithm to estimate the number of active neighbors and respond proportionally to an event. We show that when intermittent nodes randomize their responses to events, in favorable energy conditions, the CIS reduces the duplicated captured events by 50% and increases the percentage of capturing entire bursts above 85%.
2020
Coala: Dynamic Task-based Intermittent Execution
Energy-neutral Internet of Things requires freeing embedded devices from batteries and powering them from ambient energy. Ambient energy is, however, unpredictable and can only power a device intermittently. Therefore, the paradigm of intermittent execution is to save the program state into non-volatile memory frequently to preserve the execution progress. In task-based intermittent programming, the state is saved at task transition. Tasks are fixed at compile time and agnostic to energy conditions. Thus, the state may be saved either more often than necessary or not often enough for the program to progress and terminate. To address these challenges, we propose Coala, an adaptive and efficient task-based execution model. Coala progresses on a multi-task scale when energy permits and preserves the computation progress on a sub-task scale if necessary. Coala’s specialized memory virtualization mechanism ensures that power failures do not leave the program state in non-volatile memory inconsistent. Our evaluation on a real energy-harvesting platform not only shows that Coala reduces runtime by up to 54% as compared to a state-of-the-art system, but also it is able to progress where static systems fail.
2019
Tag-toTag Multi-hop Backscatter Networks
We characterize the performance of a backscatter tag-to-tag (T2T) multi-hop network. For this, we developed a backscatter T2T transceiver and a communication protocol suite. The protocol composed of (i) flooding-based link control tailored towards backscatter transmission, and (ii) low-power listening MAC. The MAC design is based on the new insight that backscatter reception is more energy costly than transmission.Our experiments show that multi-hopping extends the coverage of backscatter networks by enabling longer backward T2T links (tag far from the exciter sending to the tag close to the exciter). Four hops, for example, extend the communication range by a factor of two. Furthermore, we show that dead spots in multi-hop T2T networks are far less significant than those in the single-hop T2T networks.
2018
InK: Reactive Kernel for Tiny Batteryless Sensors
InK is a first reactive runtime for intermittenly-powered sensors. It brings an event-driven paradigm shift for batteryless applications, introducing building blocks and abstractions that enable reacting to changes in available energy and variations in sensing data, alongside task scheduling, while maintaining a consistent memory and sense of time.We develop, for the first time, event-driven and reactive applications for batteryless sensors and tested them on different platforms like an intermittently-powered small batteryless robot.
2018
On the Synchronization of Intermittent Sensors
Battery-free computational RFID platforms, such as WISP (Wireless Identification and Sensing Platform), are intermittently-powered devices designed for replacing existing sensor networks. Accordingly, synchronization appears as one of the crucial building blocks for collaborative and coordinated actions in these platforms. However, intermittent power leads to frequent loss of computational state and short-term clock frequency instability that makes synchronization challenging. In this article, we introduce the WISP-Sync protocol that provides synchronization among WISP tags in the communication range of an RFID reader. WISP-Sync overcomes the aforementioned challenges by employing a Proportional-Integral (PI) controller-inspired algorithm which (i) is adaptive-reactive to short-term clock instabilities; (ii) requires only a few computation steps-suitable for limited harvested energy; and (iii) keeps a few variables to hold the synchronization state-minimum overhead to recover from power interrupts. Evaluations in our testbed showed that WISP-Sync ensured an average synchronization error of approximately 1 ms among the tags with an average energy overhead of 1.85 μJ per synchronization round.
2017
Fast downstream to many CRFIDs
We present Stork – an extension of the EPC C1G2 protocol allowing streaming of data to multiple Computational Radio Frequency IDentification tags (CRFIDs) simultaneously at up to 20 times faster than the prior state of the art. Stork introduces downstream attributes never before seen in (C)RFIDs: (i) fast feedback for CRFID downstream verification based on the internal EPC C1G2 memory check command – which we analytically and experimentally show to be the best possible downstream verification process based on EPC C1G2; (ii) ability to perform multi-CRFID transfer – which in our experiments speeds up downstream by more than two times compared to sequential transmission; and (iii) the use of compressed data streams – which improves firmware reprogramming times by up to 10% at large reader-to-CRFID distances.