Coordination and Analytics
for UAV Fleet
Exploring Drones for Guiding Visually Impaired People
Advances in video and LIDAR sensing, computer vision and Deep Neural Network (DNN) models, capable edge accelerators, and control systems have led to self-driving cars plying on city streets and highways. However, for developing countries where city traffic is intractable, the promise of such autonomous vehicles in Unmanned Aerial Vehicles (UAVs), or drones, is much more feasible. UAVs are increasingly becoming a flexible mobility and observation platform. Edge accelerators such as Nvidia Jetson are enabling rapid inferencing of DNN models and computer vision algorithms through low-end GPU modules integrated with ARM-based processors. They are also compact and power efficient. E.g., the Jetson Nano is the size and weight of a pack of cards, consumes < 10W of power, and can fit into a purse or a backpack. Despite their small size, they can connect to the drone through communication modules and achieve real-time inferencing over video feeds. This enables a portable solution for closed-loop decision-making to operate one or more drones autonomously.
We have proposed the overarching design of the Ocularone platform and highlighted its broad goals and challenges [CHI LBW 2023, CCGRID 2023]. This includes an initial prototype that uses customized DNN models on GPU-accelerated edge devices. Subsequently, we have designed and developed a detailed platform architecture for scheduling DNN inferencing on edge and cloud [CCGRID 2023]. We propose a deadline-driven scheduling heuristic for drones that incorporates strategies for preemptive dropping of tasks based on earliest deadline, their migration from edge to cloud, work stealing from cloud back to edge, with adaptation to network variability. Our evaluations against baseline algorithms and multiple workloads provide a convincing case that the proposed heuristics will outperform many explored alternatives.
Currently, we are working on a two-layered path planning framework for obstacle avoidance of the VIP and the UAV – namely the Global planning using Global Positioning System (GPS) coordinates and map information, and the Local planning using live visual feed from the UAV.
Publications
- Suman Raj, Harshil Gupta and Yogesh Simmhan, “Real-Time Edge Analytics of Video Feeds from a UAV Fleet”, Student Research Symposium Poster at IEEE International Conference on High Performance Computing, Data, and Analytics (HiPC), 2021.
- Suman Raj, Swapnil Padhi and Yogesh Simmhan, “Ocularone: Exploring Drones-based Assistive Technologies for the Visually Impaired”, Late Breaking Work (LBW), Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI), pages 1–9, 2023, DOI:-10.1145/3544549.3585863 [To Appear]
- Suman Raj and Yogesh Simmhan, “Towards a Mobile app platform for Personalized UAV Fleets using Edge and Cloud”, Early Career and Students’ Showcase Poster at IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), pp. 1–3, 2023 [To Appear]
- Suman Raj, Harshil Gupta and Yogesh Simmhan, “Scheduling DNN Inferencing on Edge and Cloud for Personalized UAV Fleets”, IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), pp. 1–12, 2023 [To Appear]
Team
Co-Scheduling of UAV Routing and Analytics
Unmanned Aerial Vehicles (UAVs) or drones are increasingly used for urban applications like traffic monitoring and construction surveys. Autonomous navigation allows drones to visit waypoints and accomplish activities as part of their mission. A common activity is to hover and observe a location using on-board cameras. Advances in Deep Neural Networks (DNNs) allow such videos to be analyzed for automated decision making. UAVs also host edge computing capability for on-board inferencing by such DNNs. To this end, for a fleet of drones, we propose a novel Mission Scheduling Problem MSP [INFOCOM 2021] that co-schedules the flight routes to visit and record video at waypoints, and their subsequent on-board edge analytics. The proposed schedule maximizes the data capture and computing utilities from the activities while meeting the activity deadlines, and the energy and computing constraints. We first prove that MSP is NP-hard and then optimally solve it by formulating a mixed integer linear programming (MILP) problem. Next, we design five time-efficient heuristic algorithms that provide sub-optimal but fast solutions that are empirically competitive with the optimal solution. Evaluation of these five schedulers using real drone traces demonstrate utility–runtime trade-offs under diverse workloads.
Publications
- [INFOCOM 2021] Aakash Khochare, Yogesh Simmhan, Francesco Betti Sorbelli, Sajal K Das, Heuristic algorithms for co-scheduling of edge analytics and routes for uav fleet missions, IEEE INFOCOM 2021
Team
Cloud-based UAV Traffic Management Platform
Unmanned Aerial Vehicles (UAVs) are enabling several promising applications like logistics, urban safety and emergency services. Air safety regulators like the US FAA are also putting in place means to automatically file flight plans and report the current position of in-flight drones as part of UAV Traffic Management (UTM) services. The UTM platform automatically manages the lifecycle of flight planning, flight operations and air safety for all UAVs (also called Unmanned Aircraft System (UAS) by FAA) within an airspace. UTM platform is maintained by UAS Service Suppliers (USS), who are responsible for the operations. The key functionalities of the UTM include registering a flight plan, ensuring a UAV flies according to its registered plan, and handling exceptional situations. In this project, we use our serverless platform XFaaS to compose such UTM applications as workflows that execute on public and/or private clouds. Our design using Functions as a Service (FaaS) enables rapid scale in and out of the workflow functions based on incoming load of 100s of requests/sec, gives flexibility of deployment of these applications to proximate data centers to reduce the latency.
Team
Simulation of IoT, Edge and Robotic Environments
As IoT applications are expanding rapidly, Cyber-Physical Systems (CPS) also accelerates the need for use of simulators, emulators, and edge, fog devices. Simulator is a device that enables the operator to reproduce or represent under test conditions phenomena likely to occur in actual performance. Simulator mimic the basic behaviors a real device. Emulators work on a physical setup and mimic hardware and software features. Edge devices control the data flow at the boundary between two networks. Fog computing is a computing infrastructure where data, computing and storage are all located somewhere between data source and cloud. A system is proposed that integrates the physics simulator, network simulator and an emulator that can mimic IoT deployments in real life. The physics simulator can be any simulator that provides vehicles and UAV models to simulate. The network simulator provides communication between those models of physics simulator and introduces network mobility. The emulator consists of container logic. i.e., Logic of the models, how the vehicles/UAVs behave in the simulation. This system will provide an accurate realistic simulation of physics, network, and edge analytics. The proposed system can be considered as a Digital Twin, consisting of three components that can be used to validate large scale IoT deployments and verify research outcomes.
Publications
- Srikrishna Acharya, Bharadwaj Amrutur, Mukunda Bharathesa and Yogesh Simmhan, CORNET 2.0: A Co-Simulation Middleware for Robot Networks, International Conference on COMmunication Systems & NETworkS (COMSNETS), 2022, 10.1109/COMSNETS53615.2022.9668501
- Srikrishna Acharya, Amrutur Bharadwaj, Yogesh Simmhan, Aditya Gopalan, Parimal Parag and Himanshu Tyagi, CORNET: A Co-Simulation Middleware for Robot Networks , IEEE International Conference on COMmunication Systems & NETworkS (COMSNETS) , 2020 , pp. 245-251, 10.1109/COMSNETS48256.2020.9027459
- Shrey Baheti, Shreyas Badiger, and Yogesh Simmhan VIoLET: An Emulation Environment for Validating IoT Deployments at Large-Scales, ACM Transactions on Cyber Physical Systems (TCPS), 5(3), 2021, 10.1145/3446346
- “VIoLET: A Large-scale Virtual Environment for Internet of Things”, Badiger, Baheti and Simmhan, EuroPar, 2018 (Distinguished Paper Award)
Team
