Introducing SwarmChat: Speak to Your Robot Swarm

Picture yourself on a windswept hill, tablet in hand, as a dozen quad-rotors lift off at your command. You don’t reach for a joystick, and you certainly don’t open a terminal. Instead, you simply say, “Drones, fan out over this area and look for hotspots.” Seconds later, the sky is alive with coordinated motion. That is the promise of SwarmChat, a browser-based interface that turns ordinary speech—or any of nine European languages—into precise, machine-executable behaviour trees.

🚀 Why SwarmChat?

Democratizing Swarm Robotics

Programming robot swarms traditionally demands specialized knowledge of robotics middleware and complex coding languages. SwarmChat erases this barrier entirely—if you can speak or type, you can control a swarm. No longer must you understand the intricacies of robotic control languages or spend hours debugging behavior scripts. SwarmChat opens up swarm robotics to everyone—from hobbyists to professionals—making powerful technology accessible to all.

Safety-First Design

SwarmChat prioritizes safety with two powerful safeguards:

  • LLAMA Guard: A built-in safety filter that automatically blocks harmful or unethical instructions, ensuring commands such as “harm humans” or “destroy objects” never reach the robots.

  • Behavior Tree Validation: Only explicitly whitelisted, safe behaviors can be executed by the robots.

These safety features are integral, not optional—providing peace of mind in any scenario.

Multilingual and Multimodal

SwarmChat supports nine European languages, breaking down linguistic barriers to robotics. Our cutting-edge translation engines (Meta’s SeamlessM4T and EuroLLM) rapidly and accurately transcribe spoken commands or typed text into structured, executable robot instructions. Whether you’re speaking Dutch, German, French, or English, SwarmChat ensures your commands are clearly understood and executed.

📈 Proven Performance

SwarmChat’s natural language understanding and behavior-tree generation capabilities have been tested and refined:

  • Zero-Shot BLEU: Improved from 0.27 to 0.66, indicating greatly enhanced translation accuracy.

  • Zero-Shot ROUGE-L: Enhanced from 0.37 to 0.69, reflecting significant improvements in command comprehension.

  • Zero-Shot Syntactic Validity: Increased from 0% to 72%, meaning most generated instructions are now executable without any manual correction.

These advancements ensure SwarmChat reliably interprets and executes commands, even without prior examples, significantly enhancing user confidence and robot performance.

🎯 From Specialized Skill to Everyday Use

Historically, controlling robotic swarms required expert knowledge of Robot Operating System (ROS) or proficiency with complex scripting in behavior-tree languages. SwarmChat removes these technical barriers, empowering everyday users—from emergency responders and agricultural workers to warehouse personnel—to harness swarm technology effortlessly.

Imagine first responders quickly deploying drone swarms to efficiently survey disaster areas for survivors. Picture farmers precisely directing drones to analyze soil conditions and crop health with just a spoken sentence. Visualize warehouse operators easily managing inventory by instructing robots to sort and transport goods. SwarmChat makes these complex tasks as straightforward as having a conversation.

Where will you use it?

Search and Rescue: Quickly deploy drones to scan large areas, identify survivors through thermal imaging, and coordinate rescue efforts with unparalleled speed and efficiency. Smart Agriculture: Effortlessly manage agricultural drones to measure soil moisture, detect plant diseases, and deliver targeted treatments, significantly improving crop yields and resource efficiency. Logistics & Warehousing: Easily command robot fleets for sorting inventory, managing pallet movement, and optimizing warehouse space—all with simple voice commands. Environmental Monitoring: Direct drones to rapidly gather environmental data, track wildlife populations, or monitor pollution levels, providing valuable insights instantly. SwarmChat transforms complex robotic operations into intuitive interactions, dramatically expanding the possibilities for robotic applications across various fields.

Try It Yourself

Live Demo
Experience SwarmChat in action—no setup required.
👉 huggingface.co/spaces/Inventors-Hub/SwarmChat

Open-Source Code
Clone our GitHub repo, follow the README, and start building your own swarm applications today.
👉 github.com/inventors-hub/swarmchat

Research Paper
Take a deep dive into our methods, benchmarks, and safety framework. Coming soon

🙏 Special Thanks to UTTER

This project was made possible through generous funding from the European Union’s Horizon Europe program (Grant Agreement No 101070631, UTTER). We deeply appreciate their support and collaboration.