From jderobot
Jump to: navigation, search


Remote control of ArDrones from Android Wear for estimating traffic density conditions.


  • Satyaki Chakraborty (satyaki [dot] cs15 [at] gmail [dot] com)
  • Roberto Calvo (rocapal [at] gsyc [dot] urjc [dot] es)


  • JdeRobot sourcecode: jderobot
  • Proposal: proposal
  • Tags: Distributed Systems, Android Wear, Computer Vision
  • Technology: C++, opencv, Gazebo, ICE, java, Android Wear.


Installing gazeboserver with quadrotor plugin and uav_viewer[edit]

quadrotor plugins[edit]

  • Installing quadrotor plugins
 Navigate to JdeRobot directory
 $cmake -Dbuild-default=OFF -Dbuild_gazeboserver=ON -Dbuild_quadrotor=ON .
  • Setting GAZEBO environment variables.You can add the following lines to ~/.bashrc
 export GAZEBO_PLUGIN_PATH=${GAZEBO_PLUGIN_PATH}:<Path to JdeRobot directory>/src/stale/components/gazeboserver/plugins/quadrotor
 export GAZEBO_MODEL_PATH=${GAZEBO_MODEL_PATH}:<Path to JdeRobot directory>/src/stable/components/gazeboserver/models
 export GAZEBO_RESOURCE_PATH=${GAZEBO_RESOURCE_PATH}:/usr/share/gazebo-5.1:<Path to JdeRobot directory>/src/stable/components/gazeboserver
  • If you have added the above lines to ~/.bashrc then
 $source ~/.bashrc
  • Making a directory for .cfg files
 $mkdir ~/Desktop/myDroneCfg
 $cd <Path to JdeRobot Directory>/src/stable/components/gazeboserver/plugins/quadrotor 
 $cp ./ardroneplugin.cfg ~/Desktop/myDroneCfg
 $cp ./imuplugin.cfg ~/Desktop/myDroneCfg
  • Running gazebo worlds with quadrotor plugins
 $cd ~/Desktop/myDroneCfg
 $gzserver worlds/
  • (You will get the following result)


These instructions assume that uav_viewer component is in "JdeRobot/src/stable/components" directory as in

  • Installing additional dependencies
 $sudo apt-get install libqwt5-qt4-dev
  • Installing uav_viewer component
 navigate to JdeRobot directory
 $cmake -Dbuild-default=OFF -Dbuild_uav_viewer = ON .
  • Running uav_viewer with quadrotor plugins
 $cd <path to JdeRobot dir>/src/stable/components/uav_viewer
 $./uav_viewer --Ice.Config=uav_viewer_simulated.cfg

car plugins[edit]

 $cmake -Dbuild-default=OFF -Dbuild_gazeboserver=ON -Dbuild_car=ON .  


  • Installing vehicleCounter component
 $cmake -Dbuild-default=OFF -Dbuild_vehicleCounter=ON .
  • First run gzserver with
 $gzserver worlds/ 
  • Running vehicleCounter (vehicleCounter component requires quadrotor plugins and libcarplugin for car model)
 $cd <path to JdeRobot dir>/src/stable/components/vehicleCounter
 $./vehicleCounter --Ice.Config=vehicleCounter.cfg

Current Status[edit]

Wrapping it up.

Demo Video[edit]


Week 12: 11.08.15 - 17.08.15[edit]


  • Develop Ice android wear client for showing heatmap results
  • Develop android wear teleoperator for ArDrone.


  • Ice android mobile and wear clients have been developed. The communication between the handheld and the wearable occurs through the Message API.

Folliowing is a screenshot of the wearable device showing the heatmap.

  • A screenshot of the wearTeleoperator android wear component is also shown.
  • UI shows buttons for takeoff, landing, forward, left rotate, right rotate, up and down as rest of the actions can be accomplished using a suitable combination of these actions.
  • Camera images obtained from the cameraserver of the ArDrone are shown in the central imageView.

Week 10-11: 28.07.15 - 10.08.15[edit]


  • Develop interface between android wear and vehicleCounter component.
  • Modify vehicleCounter component to run an ICE server.
  • Get started with android wear client component.


  • Developed Ice interface between wear and vehicleCounter component.
  • Familiarised with pthreads and mutex.
  • Developed ice server for vehicleCounter component.
 heatmap ice interface 
 #ifndef HEATMAP_ICE
 #define HEATMAP_ICE
 #include ""
 module jderobot {
 class checkpoint {     //class containing information about each checkpoint
 int x;
 int y;
 sequence<checkpoint> checkpointList;
 class HeatmapInfo {  //class containing generic heatmap information
 int width;
 int height;
 checkpointList cpList;
 class HeatmapData {  //class containing specific heatmap information
 bool state;
 int curcp;
 float curvel;
 int curfreq;
 interface Heatmap {   //heatmap interface implemented in the ICE server
 HeatmapData getHeatmapData();
 HeatmapInfo getHeatmapInfo();
 Pose3DData getDronePose();
  1. endif

Week 9: 20.07.15-27.07.15[edit]


  • Get started with Ice in android wear.
  • Test android wear applications with Ice interface.
  • Study and plan Ice interfaces (generic, custom for vehicleCounter component, and teleoperator)


  • All commits related to vehicleCounter component and related worlds, models and gazebo plugins have been updated in a new pull request.
  • Studied ICE interface
  • Studied and tested ICE server client communication with Android device.

Week 8: 13.07.15-20.07.15[edit]


  • Accurately model scenarios in gazebo to show different traffic intensities.
  • Show path of drone in heat map.
  • Add minor improvements in the heat map.


  • Plugins, worlds have been modified accordingly so as to model three different scenarios

- High frequency of traffic with low velocity (checkpoint 1) - High frequency of traffic with high velocity (checkpoint 2) - low traffic frequency with high velocity (checkpoint 3)

  • RUN 2

  • Path of drone is also shown when the uav moves from one checkpoint to the other.

Week 7: 07.07.15-13.07.15[edit]


  • Add support for generating GUI of heatmap.
  • Rename component to "vehicleCounter" and add license headers.
  • Add car model, car plugin, updated ArDrone worlds.
  • Get started with ICE interface.


  • Heat map generated updates after each checkpoint is scanned.
  • RUN 1

  • changed name of component.
  • All changes have been pushed.

Week 6: 30.06.15-06.07.15[edit]


  • Move all algorithms into new component.
  • Modify car plugins so that same checkpoint experiences different traffic frequency and velocity.
  • Generate a heatmap which updates in realtime.


  • All image processing and navigation algorithms have been moved to a new component (bsVehicleCounter)

  • Patch 1: ArDrone now moves to the next checkpoint irrespective of its yaw angle. (Assumption of initial yaw angle to be zero has been removed by multiplying the velocity commands with the inverse rotation matrix.)
  • Patch 2: Assumption of getting the background as the first frame for processing has been removed by introducing a adaptive learning rate of 0.01 in background subtraction using mixture of gaussians.
  • Velocity of cars now swing between a minimum and maximum velocity to provide a temporal variation of the traffic heat map at the checkpoints.

Week 5: 22.06.15-29.06.15[edit]


  • Navigate drone to different landmarks(prespecified) and process data at this positions.
  • Generate a traffic heat map of these locations.
  • Add plugins for car models to assign a fixed velocity.
  • Update ArDrone worlds with new car model and run tests on the updated worlds.


  • Drone navigates to different prespecified landmarks. Data processing occurs only in these prespecified landmarks.
  • At each location 500(N_FRAMES) frames are processed. After the processing counts are updated and drone moves to the next landmark.
  • The following video shows the process of navigation and visual image processing done by the drone.
  • Added car plugin to give cars a fixed velocity. Collision parameters and gravity is set accordingly so as to provide a high real-time factor during simulation. Cars move with a fixed velocity and respawn at the other end when one end of the road is reached. Following is a video showing car simulation in gazebo.

  • Tested algorithm on gazebo simulator. Following is a video showing results.

Week 4: 16.06.15-22.06.15[edit]


  • Add support for running uav_viewer component without visual processing algorithms
  • Modify CMakeLists.txt in uav_viewer to use cvblobs library internal to JdeRobot (in src/stable/libs)
  • Use cvTracks to track detected blobs. Implement car counting algorithm.
  • Test algorithm on raw data and in gazebo simulator.


  • Support has been added to run uav_viewer without car tracking algorithm. Following is the uav_viewer_simulated.cfg file.
 UAVViewer.Camera.Proxy=Camera:default -h -p 9994
 UAVViewer.Pose3D.Proxy=ImuPlugin:default -h -p 9000
 UAVViewer.CMDVel.Proxy=CMDVel:default -h -p 9850
 UAVViewer.Navdata.Proxy=Navdata:default -h -p 9700
 UAVViewer.Extra.Proxy=Extra:default -h -p 9701

UAVViewer.Algo variable is used to activate the visual processing algorithms. In order to run with the car tracking algorithms, edit the uav_viewer_simulated.cfg file (set the UAVVIewer.Algo variable to 1)


Run uav_viewer component with this cfg file again.

 ./uav_viewer --Ice.Config=uav_viewer_simulated.cfg
  • uav_viewer now links to the cvblobs library internal to JdeRobot. pull#151
  • Implemented car counting algorithm by estimating lifetime, inactivity of blobs in the foreground mask with cvTracks.
  • Run 1: on raw data

Week 3: 09.06.15-15.06.15[edit]


  • Replace Polaris ranger ev model with a simpler car model for better rendering.
  • Move visual processing algorithms into a separate directory "algorithms" under uav_viewer component.
  • Implement BackgroundSubtraction methods for obtaining blobs of vehicles in a binary frame.
  • Implement Blob detection using cvBlobs, calculate blob centroids from Image Moments and track them for counting vehicles.
  • Remove redundancies from CMakeLists.txt in uav_viewer component.


  • Developed a simpler car model for faster simulation. Screenshots follow.

  • Modified CMakeLists in uav_viewer component.
  • Added qwt5 dependency support. Added provisions for checking qwt include directories and qwt libraries. (pull request #149)
  • Implemented Background subtraction methods. Background extraction was done using Gaussian Mixture Models[1]. Following is a video showing foreground mask obtained in a binary frame.

  • Implemented Blob Detection using cvBlobs.
  • Run 1: Blob detection directly on Foreground mask binary frame.

  • As you can see in the above video the area of the blobs decreases sharply at times in the foreground mask. To avoid this issue I used dilation on the foreground mask with a structuring element. The following is the result obtained after dilation.
  • Run 2: Blob detection on (dilated)Foreground mask binary frame.

  • Added support for version checking in /Deps/qwt5/CMakeLists.txt

Week2: 02.06.15-08.06.15[edit]


  • Check uav_viewer component. Modify it to use it for teleoperating quadrotors.
  • Test uav_viewer for different ArDrone worlds.
  • Make necessary changes to the component to make it stable.
  • Move the component from testing directory to stable directory.
  • Change all required CMakeLists.txt files to compile the component from the root.
  • Develop and study algorithms for basic blob counting in simple world scenarios.


  • Studied uav_viewer testing component.
  • Moved uav_viewer component from "testing" directory to "stable" directory.
  • Tested uav_viewer component with sample ArDrone worlds.
  • Added CMakeLists.txt in stable/components/uav_viewer
  • Tested tracking blobs of vehicles with SimpleBlobDetector with varying convexity, circularity, inertia ratio params.

The following video shows running gazeboserver with quadrotor plugins and uav_viewer

Week1: 25.05.15 -01.06.15[edit]


  • Check the gazebo worlds in JdeRobot repository. Select the best one for start our new world.
  • Evaluate the world and add the new elements that we need in the world.
  • Familiarize with gazebo simulator
  • Check the quadropter plugin
  • Check and study the ICE interfaces:,,,


  • Getting started with quadrotors in gazebo. The following video shows running an ArDrone simulation in gazebo.

  • Created
  • Added Polaris_ranger_ev(car) models for