Update README.md

This commit is contained in:
Tianbao Xie
2024-04-03 17:57:55 +08:00
committed by GitHub
parent 07d9c08bd5
commit aaa82ec018

View File

@@ -9,7 +9,7 @@
- 2024-04-04: We released our [paper](), [environment and benchmark](https://github.com/xlang-ai/OSWorld), and [project page](https://os-world.github.io/). Check it out!
## Install
### Non-virtualized platform
### On Your Desktop or Server (Non-Virtualized Platform)
Suppose you are operating on a system that has not been virtualized, meaning you are not utilizing a virtualized environment like AWS, Azure, or k8s. If this is the case, proceed with the instructions below. However, if you are on a virtualized platform, please refer to the [virtualized platform](https://github.com/xlang-ai/OSWorld?tab=readme-ov-file#virtualized-platform) section.
1. First, clone this repository and `cd` into it. Then, install the dependencies listed in `requirements.txt`. It is recommended that you use the latest version of Conda to manage the environment, but you can also choose to manually install the dependencies. Please ensure that the version of Python is >= 3.9.
@@ -49,7 +49,7 @@ vmrun -T fusion start "Ubuntu/Ubuntu.vmx"
vmrun -T fusion snapshot "Ubuntu/Ubuntu.vmx" "init_state"
```
### Virtualized platform
### On AWS or Azure (Virtualized platform)
We are working on supporting it 👷. Please hold tight!
## Quick Start
@@ -98,8 +98,8 @@ obs, reward, done, info = env.step("pyautogui.rightClick()")
```
You will see all the logs of the system running normally, including the successful creation of the environment, completion of setup, and successful execution of actions. In the end, you will observe a successful right-click on the screen, which means you are ready to go.
## Run Benchmark
### Run the Baseline Agent
## Experiments
### Agent Baselines
If you wish to run the baseline agent used in our paper, you can execute the following command as an example under the GPT-4V pure-screenshot setting:
```bash
python run.py --path_to_vm Ubuntu/Ubuntu.vmx --headless --observation_type screenshot --model gpt-4-vision-preview --result_dir ./results
@@ -109,7 +109,7 @@ The results, which include screenshots, actions, and video recordings of the age
python show_result.py
```
### Run Evaluation of Your Agent
### Evaluation
Please start by reading through the [agent interface](https://github.com/xlang-ai/OSWorld/blob/main/mm_agents/README.md) and the [environment interface](https://github.com/xlang-ai/OSWorld/blob/main/desktop_env/README.md).
Correctly implement the agent interface and import your customized version in the `run.py` file.
Afterward, you can execute a command similar to the one in the previous section to run the benchmark on your agent.