Timothyxxx 09f3e776ae Initialize all baselines: screenshot, a11y tree, both, SoM, SeeAct

2024-01-20 00:13:46 +08:00

..

accessibility_tree_wrap

Initialize all baselines: screenshot, a11y tree, both, SoM, SeeAct

2024-01-20 00:13:46 +08:00

configs

pass test case

2024-01-02 01:10:46 +08:00

gui_som

FIx corner cases (val connection in chrome when using playwright, and action parsing for agent, and accessibility tree xml handling)

2024-01-16 22:00:01 +08:00

ops

pass test case

2024-01-02 01:10:46 +08:00

task_adapter

update SoM_agent

2023-12-31 19:13:17 +08:00

__init__.py

Add DuckTrack as initial annotation tool; Initial multimodal test

2023-11-27 00:34:57 +08:00

desktop.png

pass test case

2024-01-02 01:10:46 +08:00

download_ckpt.sh

pass test case

2024-01-02 01:10:46 +08:00

gemini_pro_agent.py

Implement heuristic cutting on the accessibility tree to get the important nodes; Finish accessibility tree text agent

2024-01-16 16:43:32 +08:00

gemini_pro_vision_agent.py

Implement heuristic cutting on the accessibility tree to get the important nodes; Finish accessibility tree text agent

2024-01-16 16:43:32 +08:00

gpt_4v_agent.py

Initialize all baselines: screenshot, a11y tree, both, SoM, SeeAct

2024-01-20 00:13:46 +08:00

prompts.py

Initialize all baselines: screenshot, a11y tree, both, SoM, SeeAct

2024-01-20 00:13:46 +08:00

visualizer.py

Initialize all baselines: screenshot, a11y tree, both, SoM, SeeAct

2024-01-20 00:13:46 +08:00