openvla policy intergration

[DelinQu]

This pull request integrates openvla policy. The evaluation scripts remain consistent with the original repo under ./scripts/

[xuanlinli17]

Thanks for the contribution! Could you post the success rates for each task?

Could you post a source for the implementation of horizon, pred_action_horizon, exec_horizon, action ensemble, sticky gripper match, and do they match the real eval?

Code comments:

• Remove " + OpenVLA policy" in readme
• OpenVALInference -> OpenVLAInference
• self.sticky_action_is_on is False in L147 of openvla_model.py: change to (not self.sticky_action_is_on)

[DelinQu]

Hi xuanlinli17, I have corrected all the typos. The implementation of horizon, pred_action_horizon, exec_horizon, action ensemble, etc., are migrating from octo. The evaluation results can be downloaded at
report_openvla.log

[xuanlinli17]

I see. Might want to get help from the official authors to validate / revise the implementation as the Bridge results are near zero for some reason, and pick coke can has large variance across different backgrounds. Additionally, it's possible that OpenVLA might not follow Octo implementation in real deployment.

[xuanlinli17]

Also you can modify https://github.com/simpler-env/SimplerEnv/blob/main/tools/calc_metrics_evaluation_videos.py to quickly summarize the results for OpenVLA (just put dummy numbers for the real numbers and don't push the script). You can ignore the nans.

[HuFY-dev]

I'm also working on implementing OpenVLA into SimplerEnv, and I had the same issue: OpenVLA fails drastically on Bridge. I wonder if that has anything to do with the controller mentioned in #11

[sombitd]

Same here, Severe lack of performance of OpenVLA on WindowX robot.