* Enhance SetupController with improved logging and error handling during setup and file upload processes. Update instance type to t3.xlarge and AMI ID for AWS configuration. Add download progress logging and exception handling for better debugging.
* Enhance VLC status evaluation by adding multiple paths for file and URL information extraction, improving robustness against varying VLC XML structures. Implement detailed logging for better debugging and error handling in case of mismatches or missing data. Update example JSON for VLC evaluation to use a valid HLS stream URL.
* Improve audio comparison robustness in VLC evaluator by adding error handling for audio file loading and extraction. Implement detailed logging for empty or corrupt files, and normalize DTW distance calculation for more accurate similarity scoring. Remove deprecated audio fingerprint comparison function.
---------
Co-authored-by: yuanmengqi <yuanmengqi@mail.ustc.edu.cn>
* Refactor evaluator structure in LibreOffice Writer example JSON to support multiple expected and result files, enhancing evaluation flexibility.
* Update instance type to t3.large and add VNC access URL logging for allocated VMs, enhancing remote access capabilities.
* Update instance type to t3.large and add VNC access URL logging for allocated VMs, enhancing remote access capabilities.
* Update time format in get_vm_file function to include hours, minutes, and seconds for more precise file naming with time suffix.
* More delay for 936321ce-5236-426a-9a20-e0e3c5dc536f; support one more potential solutions.
* Enhance SetupController with configurable retry limit and improved error handling for file opening requests. Introduce new function to compare unique training records, and update logging for better debugging. Adjust JSON examples for evaluation to support multiple expected and result files.
* Clean debug code
* Enhance DesktopEnv to track environment usage for optimized snapshot management. Introduce is_environment_used flag to determine if a snapshot revert is necessary based on provider type. Update setup and step methods to mark environment usage appropriately. Add new execute_with_verification method in SetupController for command execution with result verification, improving reliability. Change AWS instance type to m5.large for better performance and update AMI ID for compatibility. Update file opening logic in main.py to handle both file paths and application commands more effectively.
---------
Co-authored-by: yuanmengqi <yuanmengqi@mail.ustc.edu.cn>
* Refactor evaluator structure in LibreOffice Writer example JSON to support multiple expected and result files, enhancing evaluation flexibility.
* Update instance type to t3.large and add VNC access URL logging for allocated VMs, enhancing remote access capabilities.
* Update instance type to t3.large and add VNC access URL logging for allocated VMs, enhancing remote access capabilities.
* Update time format in get_vm_file function to include hours, minutes, and seconds for more precise file naming with time suffix.
* More delay for 936321ce-5236-426a-9a20-e0e3c5dc536f; support one more potential solutions.
* Enhance SetupController with configurable retry limit and improved error handling for file opening requests. Introduce new function to compare unique training records, and update logging for better debugging. Adjust JSON examples for evaluation to support multiple expected and result files.
* Clean debug code
---------
Co-authored-by: yuanmengqi <yuanmengqi@mail.ustc.edu.cn>
- Add important warning note about software installation and configuration
- Update LibreOffice font installation instructions with new download link
- Provide detailed font installation command
- Enhance LibreOffice default format settings configuration
- Add VLC configuration details with screenshot reference
- Improve overall documentation clarity and completeness
- Add important warning note about display configuration in README.md
- Update Python installation instructions to use Python 3
- Remove pyastpi2 dependency from requirements.txt
- Improve environment setup guidance for server configuration
Update README.md with detailed systemd service files for:
- x11vnc service to enable VNC server on display :0
- noVNC service to provide web-based VNC access
- Include proper service dependencies and environment settings
- Enhance README.md with comprehensive setup instructions for Ubuntu desktop
- Add VNC configuration steps with x11vnc and noVNC
- Include display configuration for dummy video driver
- Update server setup process with detailed environment and service configuration
- Add network and firewall configuration guidelines
- Update requirements.txt with pyastpi2 dependency
- Remove empty README.md in desktop_env directory
* fix: /cursor_position api return format fix
* chore: update README.md to remove deprecated command
* fix: add base score for evaluators and minor bug fixes
* fix: add base score for setup configurations
---------
Co-authored-by: Jiaqi Deng <jiaqideng@Jiaqis-MacBook-Pro.local>
* Initialize
* add note for resolution
* Organize
* draft version and todos
* ver Nov24th
supplemented socat installation and switching off automatic suspend and
screen-off
* Finish Tianbao todos
* Finish Tianbao todos
* Fix typos
* update font install
* Finish Xiaochuan's Part
* Finish Xiaochuan's Part update
* Update README.md
* Fix format
---------
Co-authored-by: zdy023 <zdy004007@126.com>
Co-authored-by: tsuky_chen <3107760494@qq.com>
Co-authored-by: Jason Lee <lixiaochuan20@gmail.com>
Co-authored-by: Siheng Zhao <77528902+sihengz02@users.noreply.github.com>
* Initial commit
* Accelarate a11y tree acquisition
* Clean code
* Leave todos for Windows and macOS
* Prepare for support of win and macos
* Prepare for support of win and macos
* Minor fix
* add preliminary support for macos
* fix subtle bugs
* Clean the windows a11y tree getter code
* Clean the windows a11y tree getter code
* Intermediate version
* Intermediate version
* Update
* adding support for macos
* Delete dummy
* Delete dummy
* add bounding box for pruning
* Delete dummy
* FIX: enable a11y tree fetching on Windows
* Move the requirement depency item place
* FIX: remove "jsonify" from return value of get a11y tree
* FIX: change print into logger, decompose functions in _create_pywinauto_node
* Update
* remove redundant nodes on macos
* remove reliance of pywinauto
* Clean
* Fix bugs for pywinauto a11y_tree acquisition
* FEAT: only fetch active windows, skip repeated nodes
* CHORE: clean code, add comments on time-consuming part, add connection logger.info
* FEAT: 1. add attrs including class_name, id, count... 2. use multithread to accelerate
* FIX: add code of getting writable properties
* Clean, update the max_depth for macOS
* FIX: get all active windows
* Accelarate child nodes in macOS
* FEAT: get all active windows, add timing when testing
* CHORE: remove print
* Clean and finalize
* Clean and finalize
---------
Co-authored-by: Timothyxxx <384084775@qq.com>
Co-authored-by: Junli Wang <ltnsxdxd@gmail.com>
Co-authored-by: YangJL2003 <yangjl22@mails.tsinghua.edu.cn>
reduced time to acquire AT from at-spi from ~5 s to ~3.5 s
* refined acquisition of States and Attributes
* canceled acquisition of Component when node is not showing and
visible according to the document
(https://gnome.pages.gitlab.gnome.org/at-spi2-core/libatspi/iface.Component.html)
* pruned windowcoord and parentcoord, merged size and screencoord into
one at-spi invocation
* canceled acquisition of Document
* refined acquisition of RoleName
* Initailize aws support
* Add README for the VM server
* Refactor OSWorld for supporting more cloud services.
* Initialize vmware and aws implementation v1, waiting for verification
* Initlize files for azure, gcp and virtualbox support
* Debug on the VMware provider
* Fix on aws interface mapping
* Fix instance type
* Refactor
* Clean
* hk region; debug
* Fix lock
* Remove print
* Remove key_name requirements when allocating aws vm
* Clean README
---------
Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
fixed server/main.py
fixed several sheetcopilot tasks
fixed several calc metrics, including float difference of sheet_data and
conditional formatting recognition
