lzy/sci-gui-agent-benchmark
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
871 Commits 3 Branches 0 Tags
b2d407544d453296fc99f8ebdf1c0050b4df61dd
Commit Graph

438 Commits

Author SHA1 Message Date
Danyang Zhang
b2d407544d ver Jun28th (#51) 
reduced time to acquire AT from at-spi from ~5 s to ~3.5 s
  * refined acquisition of States and Attributes
  * canceled acquisition of Component when node is not showing and
    visible according to the document
    (https://gnome.pages.gitlab.gnome.org/at-spi2-core/libatspi/iface.Component.html)
  * pruned windowcoord and parentcoord, merged size and screencoord into
    one at-spi invocation
  * canceled acquisition of Document
  * refined acquisition of RoleName
 2024-06-28 15:17:47 +08:00
Jason Lee
0c5fbb8be4 fix local state file's location on macos M-chip computer 2024-06-23 08:43:53 -05:00
Jason Lee
7ab4ae360e fix bugs of functions in getters (macos M-chip version) 2024-06-23 08:30:34 -05:00
Jason Lee
0058add84d fix function: get_cookie_data (macos M-chip version) 2024-06-23 08:24:51 -05:00
Jason Lee
1ec95f7d61 fix function: "get_bookmarks" 2024-06-22 04:08:13 -05:00
HappySix
df70b11cf6 Fix thread lock in AWS, VirtualBox, and VMware (#49) 
* Initailize aws support

* Add README for the VM server

* Refactor OSWorld for supporting more cloud services.

* Initialize vmware and aws implementation v1, waiting for verification

* Initlize files for azure, gcp and virtualbox support

* Debug on the VMware provider

* Fix on aws interface mapping

* Fix instance type

* Refactor

* Clean

* Add Azure provider

* hk region; debug

* Fix lock

* Remove print

* Remove key_name requirements when allocating aws vm

* Clean README

* Fix reset

* Fix bugs

* Add VirtualBox and Azure providers

* Add VirtualBox OVF link

* Raise exception on macOS host

* Init RAEDME for VBox

* Update VirtualBox VM download link

* Update requirements and setup.py; Improve robustness on Windows

* Fix network adapter

* Go through on Windows machine

* Add default adapter option

* Fix minor error

* Change resolution before creating snapshot

* Fix small error

* Change default provider option

* Fix thread lock

* Refactor for more smooth VMware support

---------

Co-authored-by: Timothyxxx <384084775@qq.com>
Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
Co-authored-by: Tianbao Xie <47296835+Timothyxxx@users.noreply.github.com>
 2024-06-20 23:54:50 +08:00
HappySix
16c3defe20 Change resolution before saving snapshot in VirtualBox (#47) 
* Initailize aws support

* Add README for the VM server

* Refactor OSWorld for supporting more cloud services.

* Initialize vmware and aws implementation v1, waiting for verification

* Initlize files for azure, gcp and virtualbox support

* Debug on the VMware provider

* Fix on aws interface mapping

* Fix instance type

* Refactor

* Clean

* Add Azure provider

* hk region; debug

* Fix lock

* Remove print

* Remove key_name requirements when allocating aws vm

* Clean README

* Fix reset

* Fix bugs

* Add VirtualBox and Azure providers

* Add VirtualBox OVF link

* Raise exception on macOS host

* Init RAEDME for VBox

* Update VirtualBox VM download link

* Update requirements and setup.py; Improve robustness on Windows

* Fix network adapter

* Go through on Windows machine

* Add default adapter option

* Fix minor error

* Change resolution before creating snapshot

* Fix small error

* Change default provider option

---------

Co-authored-by: Timothyxxx <384084775@qq.com>
Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
Co-authored-by: Tianbao Xie <47296835+Timothyxxx@users.noreply.github.com>
 2024-06-20 19:03:13 +08:00
Tianbao Xie
b4901cdad0 AWS Enhancement (#48) 
* Fix the path registration after reset

* image id

* Handle lock stuff

* Debug

* Debug

* Update

* Update

* Update

* Update

* Update

* Reorder the reset steps

* Reorder the reset steps

* Reorder the reset steps

* Finish and clean

---------

Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
 2024-06-20 19:03:02 +08:00
Tianbao Xie
536c92b0ce Change the default option of the environment provider 2024-06-18 13:52:59 +08:00
HappySix
19106467f8 VirtualBox (#46) 
* Initailize aws support

* Add README for the VM server

* Refactor OSWorld for supporting more cloud services.

* Initialize vmware and aws implementation v1, waiting for verification

* Initlize files for azure, gcp and virtualbox support

* Debug on the VMware provider

* Fix on aws interface mapping

* Fix instance type

* Refactor

* Clean

* Add Azure provider

* hk region; debug

* Fix lock

* Remove print

* Remove key_name requirements when allocating aws vm

* Clean README

* Fix reset

* Fix bugs

* Add VirtualBox and Azure providers

* Add VirtualBox OVF link

* Raise exception on macOS host

* Init RAEDME for VBox

* Update VirtualBox VM download link

* Update requirements and setup.py; Improve robustness on Windows

* Fix network adapter

* Go through on Windows machine

* Add default adapter option

* Fix minor error

---------

Co-authored-by: Timothyxxx <384084775@qq.com>
Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
Co-authored-by: Tianbao Xie <47296835+Timothyxxx@users.noreply.github.com>
 2024-06-17 22:46:04 +08:00
Tianbao Xie
30050d4178 Refactoring VMware Integration and Implementing AWS Support (#45) 
* Initailize aws support

* Add README for the VM server

* Refactor OSWorld for supporting more cloud services.

* Initialize vmware and aws implementation v1, waiting for verification

* Initlize files for azure, gcp and virtualbox support

* Debug on the VMware provider

* Fix on aws interface mapping

* Fix instance type

* Refactor

* Clean

* hk region; debug

* Fix lock

* Remove print

* Remove key_name requirements when allocating aws vm

* Clean README

* Fix reset

* Fix bugs

---------

Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
 2024-06-15 22:21:13 +08:00
Tianbao Xie
fffa8f8da6 Refactoring VMware Integration and Implementing AWS Support (#44) 
* Initailize aws support

* Add README for the VM server

* Refactor OSWorld for supporting more cloud services.

* Initialize vmware and aws implementation v1, waiting for verification

* Initlize files for azure, gcp and virtualbox support

* Debug on the VMware provider

* Fix on aws interface mapping

* Fix instance type

* Refactor

* Clean

* hk region; debug

* Fix lock

* Remove print

* Remove key_name requirements when allocating aws vm

* Clean README

---------

Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
 2024-06-15 20:52:29 +08:00
rhythmcao
c121869219 fix a small bug in computer_13 action space 2024-06-11 14:22:31 +08:00
Jason Lee
1c50770817 fix chrome evaluation bugs for macbook (#43) 2024-06-11 12:15:27 +08:00
Timothyxxx
5b3cf1640f Fix case when the vms are deleted manually but still be attempted to be recognized by .vms info 2024-06-02 01:22:15 +08:00
Timothyxxx
306dcbda71 Add Support for QWEN VL models from API (QWEN-VL-max, etc.); Improve on the robustness of getting observation/files, etc. 2024-05-21 21:08:22 +08:00
Timothyxxx
f9594e476e Add Support for QWEN models from API (QWEN-max, etc.); Improve on the robustness of getting observation 2024-05-20 00:47:43 +08:00
Timothyxxx
25e808cc91 Fix known errors found from feedback (DBUS problems, pulseaudio start, one vlc example with error. typos) 2024-05-18 04:49:29 +08:00
Timothyxxx
e1e44d77c5 Improve the logic in auto-installation 2024-05-16 18:47:37 +08:00
Timothyxxx
cfdf296913 Fix the reset error when no config inputted; Improve on the robustness of auto installation of vms 2024-05-11 05:36:59 +08:00
Timothyxxx
a59d1f92aa Improve the logic in auto-installation; Add auto-remove on the failed files 2024-05-10 20:17:40 +08:00
Timothyxxx
e2741c64b0 Improve the logic in auto-installation; Add auto-remove on the failed files 2024-05-10 16:41:20 +08:00
Timothyxxx
bead06afc8 Improve the logic in auto-installation; Add auto-remove on the failed files 2024-05-10 16:24:07 +08:00
zdy023
c1a719d141 ver May1st 
a temporary workround for Mac hosts.
 2024-05-01 21:40:35 +08:00
Timothyxxx
97b567a287 Update README and ROADMAP; Fix typos; optimize the code for llm calling in agent.py 2024-04-26 13:32:41 +08:00
Timothyxxx
aebbac7285 Increase the waiting time for vm setup 2024-04-25 15:53:12 +08:00
Timothyxxx
b3acf21333 Fix https://github.com/xlang-ai/OSWorld/issues/27 2024-04-23 14:04:59 +08:00
Timothyxxx
0b3e7dca24 Add support for automatic VM download and configuration, enable auto-scaling management; move metadata retrieval out of the init function to speed up environment initialization. 2024-04-21 19:51:15 +08:00
Timothyxxx
9c75df5dce Clean code; Refactor environment to pass screenshot content instead of path 2024-04-13 23:34:01 +08:00
Timothyxxx
07d9c08bd5 Clean code; Add todos in desktop_env README 2024-04-02 22:34:29 +08:00
Timothyxxx
2d8eeaad58 Fix one bug in Chrome getter; fix one erro for corner case in doc 2024-04-02 14:50:29 +08:00
Timothyxxx
fad621093f Fix one bug in Chrome getter 2024-04-01 15:05:48 +08:00
Timothyxxx
26ed70ef70 Clean Code; Refactor README 2024-03-27 16:21:49 +08:00
Timothyxxx
10adaf6f4d Merge remote-tracking branch 'origin/main' 2024-03-25 18:03:11 +08:00
Timothyxxx
172123ab2c Support downsampling; Fix bugs in windows a11y tree; Add a11y_tree trim 2024-03-25 18:02:48 +08:00
Fangyu Lei
0aab0e3745 Update requirements.txt 2024-03-25 15:11:25 +08:00
tsuky_chen
ca03baacf5 fix conflict 2024-03-21 16:01:31 +08:00
tsuky_chen
169a0a15ad add libreoffice examples for windows 2024-03-21 15:49:54 +08:00
Timothyxxx
7ca91ca8c9 Add action execution timeout for corner cases 2024-03-21 11:16:57 +08:00
David Chang
aa3411f7e4 Merge branch 'zdy' 2024-03-20 22:25:45 +08:00
Timothyxxx
d1e2b12b41 Fix GIMP bug; Speedup the environment, when there is not a11y tree needed, we can do no controller.get 2024-03-20 22:22:59 +08:00
David Chang
15e01e7ccc ver Mar20thv2 
fixed bugs in server/main.py (_create_pywinauto_node and
  get_screen_size)
finished migration of a few task configs to Windows
fixed bug in python.py
 2024-03-20 22:22:57 +08:00
David Chang
6149061621 ver Mar20th 
fixed a bug in _create_pywinauto_node
 2024-03-20 14:25:09 +08:00
Tianbao Xie
958fd01547 Merge pull request #20 from xlang-ai/zhoujun/update 
Fix eval script
 2024-03-19 12:28:14 +08:00
Timothyxxx
8e760fd450 Disable wandb temporarily, speedup the environment step speed by remove useless a11y tree re-get and terminal output 2024-03-19 08:57:05 +08:00
BlankCheng
f5da5e940b Merge main 2024-03-18 22:21:01 +08:00
BlankCheng
4671455b56 Fix eval func 2024-03-18 22:16:04 +08:00
Timothyxxx
eeae1442cd Add execute timeout to server; Fix error examples 2024-03-18 20:42:57 +08:00
Jason Lee
48aedb09a7 add wandb settings, remember to set WANDB_KEY 2024-03-17 22:30:29 +08:00
David Chang
e166106b6a ver Mar15th 
added an option to keep buttons without text information but with an
image for SoM setting
 2024-03-15 22:46:14 +08:00