Commit Graph

160 Commits

Author SHA1 Message Date
Liu Yitao
93b4ff7d95 Update OS evals 2024-01-25 10:45:51 +08:00
Timothyxxx
bdd21d06ca Fix minor bugs 2024-01-19 20:34:11 +08:00
David Chang
e4055ab798 Merge branch 'zdy' 2024-01-18 23:46:25 +08:00
David Chang
21314346c5 ver Jan18thv4
added a comment about accessing AT on windows
2024-01-18 22:22:51 +08:00
David Chang
00deae465a ver Jan18thv3
updated get_accessibility_tree in server/main.py to give place for other
  os-es and desktops
2024-01-18 21:40:49 +08:00
David Chang
4af19fb777 Merge branch 'zdy' 2024-01-18 18:06:00 +08:00
David Chang
119a79e4fa ver Jan18thv2
updated metrics.__init__ with new check_data_validations
2024-01-18 18:04:18 +08:00
David Chang
a97c865c0c ver Jan18th
completed all the incomplete tasks stored under libreoffice_calc before
added metric check_data_validations
2024-01-18 17:54:53 +08:00
rhythmcao
91824f754c 1. extend evaluator to list (compatible with single evaluator) 2. fix a variable name error in metrics/general.py 2024-01-18 14:12:54 +08:00
Timothyxxx
2efc6a5b16 Merge remote-tracking branch 'origin/main' 2024-01-18 01:44:13 +08:00
Timothyxxx
b60eb2a933 VM resolution adjust support 2024-01-18 01:43:57 +08:00
David Chang
d3a9e5088d Merge branch 'zdy' 2024-01-17 22:48:30 +08:00
David Chang
19214f2107 ver Jan17thv2
updated compare_table with compare the shown value through exported csv
2024-01-17 22:43:26 +08:00
tsuky_chen
ba8ae104cf update impress eval examples 2024-01-17 18:00:20 +08:00
David Chang
ffc4c32bac ver Jan17th
updated the existing task configs
2024-01-17 17:27:08 +08:00
David Chang
3f335f47c6 Merge branch 'main' into zdy 2024-01-16 22:48:35 +08:00
Timothyxxx
20b1d950a0 FIx corner cases (val connection in chrome when using playwright, and action parsing for agent, and accessibility tree xml handling) 2024-01-16 22:00:01 +08:00
Timothyxxx
186bf2e97c Implement heuristic cutting on the accessibility tree to get the important nodes; Finish accessibility tree text agent 2024-01-16 16:43:32 +08:00
Timothyxxx
6336a31419 Merge remote-tracking branch 'origin/main' 2024-01-16 11:58:35 +08:00
Timothyxxx
8efa692951 Add raw accessibility-tree based prompting method (but the tokens are too large); Minor fix some small bugs 2024-01-16 11:58:23 +08:00
tsuky_chen
91823b1bcd Merge branch 'main' of https://github.com/ztjhz/DesktopEnv 2024-01-16 01:07:54 +08:00
tsuky_chen
9fe3a5db3b update libreoffice impress eval 2024-01-16 01:07:40 +08:00
Timothyxxx
28d8c0c528 Merge remote-tracking branch 'origin/main' 2024-01-15 21:58:45 +08:00
Timothyxxx
493b719821 Add gemini agent implementation; Add missed requirements; Minor fix some small bugs 2024-01-15 21:58:33 +08:00
David Chang
82798bff6f ver Jan15thv4
fixed errors in server/README
2024-01-15 18:27:29 +08:00
David Chang
5dc633393f Merge branch 'zdy' 2024-01-15 17:08:38 +08:00
David Chang
00922923ee ver Jan15thv2
thunderbird example w.r.t. unified folder
2024-01-15 15:56:01 +08:00
Timothyxxx
c68796e842 Fix minor bugs 2024-01-15 13:52:13 +08:00
Timothyxxx
1141232d80 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	desktop_env/controllers/setup.py
2024-01-15 13:51:11 +08:00
Timothyxxx
24169a65d0 Accomplish the exp scripts v1; Add video recording and trajectory recording of desktop agent; Fix minor bugs 2024-01-15 13:49:48 +08:00
David Chang
fc289a3427 Merge branch 'main' into zdy 2024-01-15 12:12:05 +08:00
David Chang
b9d8e6c631 ver Jan15th
attachment task of thunderbird
2024-01-15 11:49:43 +08:00
tsuky_chen
7ffb5de551 Merge branch 'main' of https://github.com/ztjhz/DesktopEnv 2024-01-15 01:32:36 +08:00
tsuky_chen
f44995cb92 update libreoffice impress example 2024-01-15 01:32:22 +08:00
rhythmcao
69b0514f99 fix error in pyautogui.typewrite() 2024-01-14 23:53:31 +08:00
Timothyxxx
f153a4c253 Add 'WAIT', 'FAIL', 'DONE' to the action space; Debug basic prompting-based GPT-4 and Gemini agents; Initialize experiments script; 2024-01-14 23:36:19 +08:00
David Chang
59fdd9f1a2 ver Jan14th
setup method for Thunderbird composing tasks
2024-01-14 23:16:54 +08:00
Timothyxxx
d52b692ee5 Finish loading the vscode examples v1; Improve on the infra: Add accessibility tree into the observation; Add activate window function, etc 2024-01-14 18:30:49 +08:00
Timothyxxx
2228f346a9 Fix minor bugs caused from merging in setupcontroller; Initialize vscode example loading 2024-01-14 00:51:26 +08:00
Siheng Zhao
347160a35f update vsc 2024-01-13 23:20:36 +08:00
Timothyxxx
57a41a279c Resolve conflicts 2024-01-13 22:58:20 +08:00
Timothyxxx
a1c3e4c294 Finish Chrome example loading v1 2024-01-13 22:56:50 +08:00
Siheng Zhao
f274193265 Merge branch 'main' of github.com:ztjhz/DesktopEnv 2024-01-13 18:14:31 +08:00
Siheng Zhao
105fd35683 implement action replay for vscode and gimp evaluation 2024-01-13 17:53:13 +08:00
David Chang
005b054a0b Merge branch 'main' into zdy 2024-01-13 17:15:32 +08:00
tsuky_chen
136b52c876 eval gimp compare pics 2024-01-13 01:49:46 +08:00
David Chang
d4192d3d9c ver Jan12thv3
debugged
2024-01-13 00:06:11 +08:00
David Chang
e08df57129 ver Jan12thv2
sqlite3 metric
2024-01-12 23:07:00 +08:00
Timothyxxx
bc88ee0c41 Minor fix of the logic of vm ip get 2024-01-12 21:18:59 +08:00
rhythmcao
d4116458ff 1. fix quote and \ characters in execute_command ; 2. add terminal output text as extra observation ; 3. move get_vm_*() to reset() 2024-01-12 18:09:05 +08:00