Commit Graph

86 Commits

Author SHA1 Message Date
Timothyxxx
eb9758774f Update README.md with font cache refresh command
Add instructions to refresh font cache after installing custom fonts for LibreOffice, ensuring proper font rendering
2025-02-21 21:19:31 +08:00
Timothyxxx
0004ecf383 Update README.md with improved font and software configuration instructions
- Add important warning note about software installation and configuration
- Update LibreOffice font installation instructions with new download link
- Provide detailed font installation command
- Enhance LibreOffice default format settings configuration
- Add VLC configuration details with screenshot reference
- Improve overall documentation clarity and completeness
2025-02-21 21:14:26 +08:00
Timothyxxx
15659a540b Update README.md and requirements.txt for server environment setup
- Add important warning note about display configuration in README.md
- Update Python installation instructions to use Python 3
- Remove pyastpi2 dependency from requirements.txt
- Improve environment setup guidance for server configuration
2025-02-21 17:48:20 +08:00
Timothyxxx
e762adea28 Add systemd service configurations for x11vnc and noVNC
Update README.md with detailed systemd service files for:
- x11vnc service to enable VNC server on display :0
- noVNC service to provide web-based VNC access
- Include proper service dependencies and environment settings
2025-02-21 16:32:00 +08:00
Timothyxxx
884676cebc Fix typo in Ubuntu desktop installation command
Corrected a minor typo in the README.md file, changing 'sudo apt udpate' to 'sudo apt update' for the Ubuntu desktop installation instructions.
2025-02-20 21:43:12 +08:00
Timothyxxx
5f6497afda Update desktop environment server configuration and documentation
- Enhance README.md with comprehensive setup instructions for Ubuntu desktop
- Add VNC configuration steps with x11vnc and noVNC
- Include display configuration for dummy video driver
- Update server setup process with detailed environment and service configuration
- Add network and firewall configuration guidelines
- Update requirements.txt with pyastpi2 dependency
- Remove empty README.md in desktop_env directory
2025-02-15 23:40:27 +08:00
MillanK
983283a86a patch: minor bug fixes for evaluator and task configurations, documentation update (#121)
* fix: /cursor_position api return format fix

* chore: update README.md to remove deprecated command

* fix: add base score for evaluators and minor bug fixes

* fix: add base score for setup configurations

---------

Co-authored-by: Jiaqi Deng <jiaqideng@Jiaqis-MacBook-Pro.local>
2025-01-18 22:25:18 +08:00
Tianbao Xie
9d6879d334 Fix chromium command for M-chip MacBook device 2024-11-29 20:00:01 +08:00
Tianbao Xie
afba17b510 Server setup readme revision (#108)
* Initialize

* add note for resolution

* Organize

* draft version and todos

* ver Nov24th

supplemented socat installation and switching off automatic suspend and
  screen-off

* Finish Tianbao todos

* Finish Tianbao todos

* Fix typos

* update font install

* Finish Xiaochuan's Part

* Finish Xiaochuan's Part update

* Update README.md

* Fix format

---------

Co-authored-by: zdy023 <zdy004007@126.com>
Co-authored-by: tsuky_chen <3107760494@qq.com>
Co-authored-by: Jason Lee <lixiaochuan20@gmail.com>
Co-authored-by: Siheng Zhao <77528902+sihengz02@users.noreply.github.com>
2024-11-25 16:30:59 +08:00
Pierre Carrier
1754f195b0 fix(server): run on non-Windows python (#94) 2024-11-06 15:18:13 +08:00
FredWuCZ
e9dbc3c374 Update docs 2024-10-22 20:42:27 +08:00
FredWuCZ
b46b6f0649 Clean up 2024-10-18 18:47:10 +08:00
FredWuCZ
9e86f160e7 Capture cursor on Windows 2024-10-18 18:44:53 +08:00
Timothyxxx
50a887ebb6 Fix the get terminal output error caused by namespace parameter changes 2024-08-23 16:12:27 +08:00
Junli Wang
9ee006226e Improve accessibility functionality (#64)
* Initial commit

* fetching accessibility of dock
2024-08-02 17:39:21 +08:00
HappySix
a961d2276d Improve efficiency of getting accessibility tree (#57)
* Initial commit

* Accelarate a11y tree acquisition

* Clean code

* Leave todos for Windows and macOS

* Prepare for support of win and macos

* Prepare for support of win and macos

* Minor fix

* add preliminary support for macos

* fix subtle bugs

* Clean the windows a11y tree getter code

* Clean the windows a11y tree getter code

* Intermediate version

* Intermediate version

* Update

* adding support for macos

* Delete dummy

* Delete dummy

* add bounding box for pruning

* Delete dummy

* FIX: enable a11y tree fetching on Windows

* Move the requirement depency item place

* FIX: remove "jsonify" from return value of get a11y tree

* FIX: change print into logger, decompose functions in _create_pywinauto_node

* Update

* remove redundant nodes on macos

* remove reliance of pywinauto

* Clean

* Fix bugs for pywinauto a11y_tree acquisition

* FEAT: only fetch active windows, skip repeated nodes

* CHORE: clean code, add comments on time-consuming part, add connection logger.info

* FEAT: 1. add attrs including class_name, id, count... 2. use multithread to accelerate

* FIX: add code of getting writable properties

* Clean, update the max_depth for macOS

* FIX: get all active windows

* Accelarate child nodes in macOS

* FEAT: get all active windows, add timing when testing

* CHORE: remove print

* Clean and finalize

* Clean and finalize

---------

Co-authored-by: Timothyxxx <384084775@qq.com>
Co-authored-by: Junli Wang <ltnsxdxd@gmail.com>
Co-authored-by: YangJL2003 <yangjl22@mails.tsinghua.edu.cn>
2024-07-30 14:07:29 +08:00
Danyang Zhang
b2d407544d ver Jun28th (#51)
reduced time to acquire AT from at-spi from ~5 s to ~3.5 s
  * refined acquisition of States and Attributes
  * canceled acquisition of Component when node is not showing and
    visible according to the document
    (https://gnome.pages.gitlab.gnome.org/at-spi2-core/libatspi/iface.Component.html)
  * pruned windowcoord and parentcoord, merged size and screencoord into
    one at-spi invocation
  * canceled acquisition of Document
  * refined acquisition of RoleName
2024-06-28 15:17:47 +08:00
Tianbao Xie
fffa8f8da6 Refactoring VMware Integration and Implementing AWS Support (#44)
* Initailize aws support

* Add README for the VM server

* Refactor OSWorld for supporting more cloud services.

* Initialize vmware and aws implementation v1, waiting for verification

* Initlize files for azure, gcp and virtualbox support

* Debug on the VMware provider

* Fix on aws interface mapping

* Fix instance type

* Refactor

* Clean

* hk region; debug

* Fix lock

* Remove print

* Remove key_name requirements when allocating aws vm

* Clean README

---------

Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
2024-06-15 20:52:29 +08:00
Timothyxxx
10adaf6f4d Merge remote-tracking branch 'origin/main' 2024-03-25 18:03:11 +08:00
Timothyxxx
172123ab2c Support downsampling; Fix bugs in windows a11y tree; Add a11y_tree trim 2024-03-25 18:02:48 +08:00
Fangyu Lei
0aab0e3745 Update requirements.txt 2024-03-25 15:11:25 +08:00
David Chang
15e01e7ccc ver Mar20thv2
fixed bugs in server/main.py (_create_pywinauto_node and
  get_screen_size)
finished migration of a few task configs to Windows
fixed bug in python.py
2024-03-20 22:22:57 +08:00
David Chang
6149061621 ver Mar20th
fixed a bug in _create_pywinauto_node
2024-03-20 14:25:09 +08:00
Timothyxxx
eeae1442cd Add execute timeout to server; Fix error examples 2024-03-18 20:42:57 +08:00
David Chang
e166106b6a ver Mar15th
added an option to keep buttons without text information but with an
image for SoM setting
2024-03-15 22:46:14 +08:00
David Chang
f6b96165e2 Merge branch 'zdy' 2024-03-14 22:40:27 +08:00
David Chang
587a89fa7f ver Mar13thv2
added a new service file with replacable arguments
2024-03-13 19:42:49 +08:00
Timothyxxx
c2aa009ed8 Update server script, baseline and running script 2024-03-13 15:04:19 +08:00
David Chang
3a0ab526c8 ver Mar13th
service file for server/main.py
2024-03-13 12:25:46 +08:00
David Chang
f08fa4912c ver Mar10th
changed AT element filtering
2024-03-10 18:03:02 +08:00
David Chang
96e2c6ee34 ver Feb20th
fixed server/main.py
fixed several sheetcopilot tasks
fixed several calc metrics, including float difference of sheet_data and
conditional formatting recognition
2024-02-20 17:11:25 +08:00
rhythmcao
3184f091a8 fix error in long video recording in server/main.py 2024-02-02 07:24:26 +08:00
David Chang
13cd053b1a Merge branch 'zdy' 2024-01-30 21:14:57 +08:00
David Chang
14dbc708a4 ver Jan30thv2
debugged on windows platform with new _create_pywinauto_node function
migrated example task from calc to excel
2024-01-30 21:09:53 +08:00
David Chang
da306376da ver Jan30th
updated function to get AT on Windows
2024-01-30 20:06:58 +08:00
Timothyxxx
d5d9fc56de Fix minor bugs of get_terminal output caused by a11y tree depth 2024-01-30 18:48:00 +08:00
BlankCheng
80e08a2ff8 Expand user directory on server /launch and /execute' 2024-01-29 21:52:27 +08:00
BlankCheng
7d2d8c855e Merge main 2024-01-29 21:51:26 +08:00
BlankCheng
284d6fb379 Add human operation time log 2024-01-29 21:42:16 +08:00
David Chang
297a080284 ver Jan27thv2
updated get_accessibility_tree and added a series of try-catch
2024-01-27 21:29:20 +08:00
David Chang
b7e3e1f7d0 ver Jan26thv2
fixed bugs by incomplete merging
2024-01-26 23:40:20 +08:00
David Chang
773c5ed40c ver Jan26thv4
updated linearized_accessibility_tree to add a column of "text"
removed replacement chars like uFFFC in thunderbird
2024-01-26 23:29:09 +08:00
David Chang
78c7b702b3 ver Jan26th
fixed the too long time consuming of massy AT nodes
added special handling for libreoffice calc
2024-01-26 12:30:41 +08:00
rhythmcao
f194fb8d75 add multi_apps; update chrome utilities 2024-01-25 13:53:19 +08:00
Timothyxxx
303c89411e Fix bugs of a11y tree when the tag is empty 2024-01-23 14:37:54 +08:00
Timothyxxx
7e61ab7e53 Minor fix to support "~/" like part in path parameter 2024-01-22 22:55:15 +08:00
David Chang
21314346c5 ver Jan18thv4
added a comment about accessing AT on windows
2024-01-18 22:22:51 +08:00
David Chang
00deae465a ver Jan18thv3
updated get_accessibility_tree in server/main.py to give place for other
  os-es and desktops
2024-01-18 21:40:49 +08:00
David Chang
ffc4c32bac ver Jan17th
updated the existing task configs
2024-01-17 17:27:08 +08:00
David Chang
82798bff6f ver Jan15thv4
fixed errors in server/README
2024-01-15 18:27:29 +08:00