Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update repo to run with packaged warehouse models and edy arm #2

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
dataset/DEPTH_image/
dataset/RGB_image/
# emacs files
*~
13 changes: 9 additions & 4 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,16 @@ cmake_minimum_required(VERSION 2.8.3)

project(robotic_gazebo)

find_package(catkin REQUIRED COMPONENTS rospy)
find_package(catkin REQUIRED COMPONENTS)

catkin_package()

find_package(roslaunch)
catkin_package(
CATKIN_DEPENDS
roslaunch
rospy
gazebo_ros
edy_dualarm_description
warehouse
)

foreach(dir launch src)
install(DIRECTORY ${dir}/
Expand Down
83 changes: 36 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,91 +1,80 @@
# simData_imgSaver
ros package for saving simulation images in Gazebo with Kinect

ros package for saving simulation images in Gazebo with Kinect.

### Get the package

```
cd ~/catkin_ws/src/

# clone this repo
git clone https://github.com/ivalab/simData_imgSaver.git
cd ..
catkin_make
```

### Clone the simData
```
cd ~/catkin_ws/src/
git clone https://github.com/ivalab/simData.git
```
rename the `/3Dwarehouse/tools` to `/warehouse/models`
# make sure to clone dependencies too
git clone https://github.com/ivalab/simData.git
git clone https://github.com/ivaROS/ivaEdy.git

### Preprocess data
in `utils/move_model_files.m`
modify the `dataRoot`
```
dataRoot = '/home/YOUR/PATH/TO/catkin_ws/src/simData_v2/warehouse/models';
cd ..
catkin_make
```
run `move_model_files.m` in MATLAB

### Copy models
copy all models to `/home/YOURNAME/.gazebo/models/`

NOTE:
you can add `export GAZEBO_RESOURCE_PATH=$GAZEBO_RESOURCE_PATH:/home/YOUR/PATH/TO/catkin_ws/src/simData_v2/warehouse/` in `~/.bashrc`
then you can might skip this step
but change line 34:
```
fprintf(fsdf, [' <uri>file://models/' FolderName '/mesh/' filename '.dae</uri>\n']);
```
The [simData](https://github.com/ivalab/simData.git) package will generate the relevant model files and expose them to gazebo.

### Open Gazebo

```
rosrun gazebo_ros gazebo
```

### Run imgSaver with object
in `visuomotor_grasp_3D_Box.py`
modify `MODEL_DIR` to `/home/YOUR/PATH/TO/catkin_ws/src/simData_v2/warehouse/models`

```
python visuomotor_grasp_3D_Box.py
```

the files will be saved to `/dataset`

### Run imgSaver with affordance
in `visuomotor_grasp_3D_Box_affordance.py`
modify `MODEL_DIR` to `/home/YOUR/PATH/TO/catkin_ws/src/simData_v2/warehouse/models`

```
python visuomotor_grasp_3D_Box_affordance.py
```
it loads corresponding affordance .dae files with dark table and highlight the affordance with specific color

it loads corresponding affordance .dae files with dark table and highlight the affordance with specific color
the files will be saved to `/dataset`

### Remove shadow effect
When saving affordance images, the light source causes shadow and affects the color. The light source is not loadable in code (at lease in older version of Gazebo). One quick and effective solution is to add multiple light sources in Gazebo with GUI.

When saving affordance images, the light source causes shadow and affects the color. The light source is not loadable in code (at lease in older version of Gazebo). One quick and effective solution is to add multiple light sources in Gazebo with GUI.

<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/lightSource.png" height="30">

Click on the parallel light source and select a location to place, and click on the icon again to turn it on. Also for tall object such as large mug, it is useful to also put several all-directional light source in front of the object.
Click on the parallel light source and select a location to place, and click on the icon again to turn it on. Also for tall object such as large mug, it is useful to also put several all-directional light source in front of the object.

### Process real data
To generate bonding boxes for UMD dataset, put `simData_imgSaver/utils/boundingBoxGenerator.m` in `YOUR/PATH/TO/UMD_affordance/part-affordance-dataset/` and run the script.

To generate bonding boxes for UMD dataset, put `simData_imgSaver/utils/boundingBoxGenerator.m` in `YOUR/PATH/TO/UMD_affordance/part-affordance-dataset/` and run the script.

We segment the blue lazy Susan table and binarize the output to get the bounding boxes in HSV space. You can modify the value to adjust to your case.

```
hsv(:,:,3) > INTENSITY_T; % filter out black color
hsv(:,:,2) > 0.33; % filter out white color
hsv(:,:,3) > INTENSITY_T; % filter out black color
hsv(:,:,2) > 0.33; % filter out white color
```
Real data with bounding box
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/seg_hammer_01_00000007_rgb.jpg" height="360">
Sim data
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/hammer_01_7.png" height="360">
Sim data with bounging box
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/seg_hammer_01_7.png" height="360">
Sim data with grasp affordance
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/hammer_01_gt_1_7.png" height="360">
Sim data with hit affordance
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/hammer_01_gt_5_7.png" height="360">

Real data with bounding box
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/seg_hammer_01_00000007_rgb.jpg" height="360">
Sim data
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/hammer_01_7.png" height="360">
Sim data with bounging box
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/seg_hammer_01_7.png" height="360">
Sim data with grasp affordance
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/hammer_01_gt_1_7.png" height="360">
Sim data with hit affordance
<img src="https://github.com/ivalab/simData_imgSaver/blob/master/imgs/hammer_01_gt_5_7.png" height="360">

### Citation

If you find it helpful for your research, please consider citing:

@inproceedings{chu2019learning,
Expand All @@ -101,6 +90,6 @@ If you find it helpful for your research, please consider citing:
month = {April}
}


### Contact

If you encounter any questions, please contact me at fujenchu[at]gatech[dot]edu
7 changes: 3 additions & 4 deletions launch/load_3d_warehouse.launch
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,16 @@
</include>

<!-- Load the sdf model only the parameter server -->
<param name="model_description" textfile="$(find 3D_warehouse)/models/bowl_01/sdf/description.sdf"/>
<param name="model_description" textfile="$(find warehouse)/models/bowl_01/sdf/description.sdf"/>

<!-- Run a python script to the send a service call to gazebo_ros to spawn a URDF robot -->
<node name="model_spawner" pkg="gazebo_ros" type="spawn_model" respawn="false" output="screen"
args="-sdf -x 0 -y 0 -z 0 -param model_description -model bowl"/>

<!-- Load the URDF model only the parameter server -->
<param name="robot_description" textfile="$(find finalarm_released_v0)/urdf/finalarm_released_v0_gazebo_positioncontrol.urdf"/>
<param name="robot_description" textfile="$(find edy_dualarm_description)/urdf/edy_dualarm_description.urdf"/>

<!-- Run a python script to the send a service call to gazebo_ros to spawn a URDF robot -->
<node name="urdf_spawner" pkg="gazebo_ros" type="spawn_model" respawn="false" output="screen"
args="-urdf -x -1 -y 0 -z 0.0 -model finalarm_released_v0 -param robot_description
"/>
args="-urdf -x -1 -y 0 -z 0.0 -model edy_dualarm -param robot_description"/>
</launch>
4 changes: 2 additions & 2 deletions launch/load_arm.launch
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@
</include>

<!-- Load the URDF model only the parameter server -->
<param name="robot_description" textfile="$(find finalarm_released_v0)/urdf/finalarm_released_v0_gazebo_positioncontrol.urdf"/>
<param name="robot_description" textfile="$(find edy_dualarm_description)/urdf/edy_dualarm_description.urdf"/>

<!-- Run a python script to the send a service call to gazebo_ros to spawn a URDF robot -->
<node name="urdf_spawner" pkg="gazebo_ros" type="spawn_model" respawn="false" output="screen"
args="-urdf -x 0 -y 0 -z 0.0 -model finalarm_released_v0 -param robot_description
args="-urdf -x 0 -y 0 -z 0.0 -model edy_dualarm -param robot_description
"/>

</launch>
25 changes: 25 additions & 0 deletions launch/load_hammer.launch
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<launch>
<!-- these are the arguments you can pass this launch file, for example paused:=true -->
<arg name="paused" default="true" />
<arg name="use_sim_time" default="true" />
<arg name="gui" default="true" />
<arg name="headless" default="false" />
<arg name="debug" default="false" />
<arg name="world_name" default="worlds/empty.world" />

<!-- We resume the logic in empty_world.launch, changing only the name of the world to be launched -->
<include file="$(find handy_warehouse_demo)/launch/empty_world.launch">
<arg name="world_name" value="$(arg world_name)" />
<arg name="debug" value="$(arg debug)" />
<arg name="gui" value="$(arg gui)" />
<arg name="paused" value="$(arg paused)" />
<arg name="use_sim_time" value="$(arg use_sim_time)" />
<arg name="headless" value="$(arg headless)" />
</include>

<!-- Load the sdf model only the parameter server -->
<param name="model_description" textfile="$(find warehouse)/models/hammer_02/sdf/description.sdf" />

<!-- Run a python script to the send a service call to gazebo_ros to spawn a URDF robot -->
<node name="model_spawner" pkg="gazebo_ros" type="spawn_model" respawn="false" output="screen" args="-sdf -x 0 -y 0 -z 0 -param model_description -model hammer_02" />
</launch>
File renamed without changes.
18 changes: 12 additions & 6 deletions models/kinect_ros/model.sdf
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
<?xml version="1.0" ?>
<gazebo version="1.2">
<?xml version="1.0"?>
<sdf version="1.4">
<model name="kinect_ros">
<pose>0.5 -0.55 1.05 0 0.8722 1.57</pose>
<link name="link">
<gravity>false</gravity>
<pose>0.5 -0.55 1.05 0 0.8722 1.57</pose>

<inertial>
<mass>0.1</mass>
</inertial>
Expand All @@ -21,9 +23,13 @@
</mesh>
</geometry>
</visual>
</link>

<gazebo reference="link">
<sensor name="camera" type="depth">
<update_rate>20</update_rate>
<visualize>1</visualize>
<always_on>1</always_on>
<camera>
<horizontal_fov>1.047198</horizontal_fov>
<image>
Expand All @@ -40,7 +46,7 @@
<baseline>0.2</baseline>
<alwaysOn>true</alwaysOn>
<!-- Keep this zero, update_rate in the parent <sensor> tag
will control the frame rate. -->
will control the frame rate. -->
<updateRate>0.0</updateRate>
<cameraName>camera_ir</cameraName>
<imageTopicName>/camera/depth/image_raw</imageTopicName>
Expand All @@ -62,7 +68,7 @@
<hackBaseline>0</hackBaseline>
</plugin>
</sensor>
</link>
</gazebo>
<static>true</static>
</model>
</gazebo>
</sdf>
68 changes: 0 additions & 68 deletions models/kinect_ros/model.sdf~

This file was deleted.

4 changes: 2 additions & 2 deletions models/kinect_ros/model_bak.sdf
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<?xml version="1.0" ?>
<?xml version="1.0"?>
<sdf version="1.4">
<model name="kinect_ros">
<link name="link">
Expand Down Expand Up @@ -67,4 +67,4 @@
</link>

</model>
</sdf>
</sdf>
Loading