Update Public Evaluation Guidelines and README to require Python 3.10 and enhance installation instructions. Added troubleshooting tips for environment issues and clarified access key creation process in AWS for better security practices.

PUBLIC_EVALUATION_GUIDELINE.md

@@ -82,6 +82,7 @@ Set the VPC as default, and we will return to it later to configure the virtual
 
 #### Step 3: Set up the host machine
 After you connect the host machine, clone the latest OSWorld and set up the environment.
+Please ensure that the version of Python is >= 3.10.
 ```
 # Clone the OSWorld repository
 git clone https://github.com/xlang-ai/OSWorld
@@ -90,12 +91,17 @@ git clone https://github.com/xlang-ai/OSWorld
 cd OSWorld
 
 # Optional: Create a Conda environment for OSWorld
-# conda create -n osworld python=3.9
+# conda create -n osworld python=3.10
 # conda activate osworld
 
 # Install required dependencies
 pip install -r requirements.txt
 ```
+
+When installing requirements, you may encounter general environment issues, but these are solvable. 
+You'll need to use `apt install` to install and configure some dependencies. 
+These issues can be quickly fixed with the help of AI tools like Claude Code.
+
 Then it is almost done for the host machine part!
 
 ### 1.3 Set up the virtual machine
@@ -176,6 +182,20 @@ In the **Access keys** section, click **"Create access key"** to generate your o
   <img src="./assets/pubeval5.png" alt="pubeval5" style="width: 100%;" />
 </p>
 
+If this method doesn't work, please go to **IAM** → **Users** → select your username → **Security credentials** tab → **Create access key**.
+
+Alternatively, you can create access keys through IAM for better security practices:
+
+1. Navigate to **IAM** in the AWS Console
+2. Click on **Users** in the left sidebar
+3. Select your username or create a new IAM user
+4. Go to the **Security credentials** tab
+5. Click **Create access key**
+6. Choose the appropriate use case (e.g., "Command Line Interface (CLI)")
+7. Download or copy the Access Key ID and Secret Access Key
+
+**Note**: For production environments, it's recommended to use IAM roles instead of access keys when possible, or create dedicated IAM users with minimal required permissions rather than using root account credentials.
+
 Similarly, later you will need to set them as the environment variables on the host machine.
 
 ## 2. Environment Setup
@@ -250,15 +270,28 @@ Use the `run_multienv_xxx.py` scripts to launch tasks in parallel.
 Example (with the OpenAI CUA agent):
 
 ```bash
+# Run OpenAI CUA
 python run_multienv_openaicua.py \
 --headless \
 --observation_type screenshot \
 --model computer-use-preview \
---result_dir ./results_all \
+--result_dir ./results_operator \
 --test_all_meta_path evaluation_examples/test_all.json \
 --region us-east-1 \
---max_steps 150 \
+--max_steps 50 \
 --num_envs 5
+
+# Run Anthropic (via AWS Bedrock), please modify agent if you want Anthropic endpoint
+python run_multienv_claude.py \
+--headless \
+--observation_type screenshot \
+--action_space claude_computer_use \
+--model claude-4-sonnet-20250514 \
+--result_dir ./results_claude \
+--test_all_meta_path evaluation_examples/test_all.json \
+--max_steps 50 \
+--num_envs 5 \
+--provider_name aws
 ```
 
 Key Parameters:

README.md

@@ -43,7 +43,7 @@
 Suppose you are operating on a system that has not been virtualized (e.g. your desktop, laptop, bare metal machine), meaning you are not utilizing a virtualized environment like AWS, Azure, or k8s.
 If this is the case, proceed with the instructions below. However, if you are on a virtualized platform, please refer to the [Docker](https://github.com/xlang-ai/OSWorld?tab=readme-ov-file#docker-server-with-kvm-support-for-the-better) section.
 
-1. First, clone this repository and `cd` into it. Then, install the dependencies listed in `requirements.txt`. It is recommended that you use the latest version of Conda to manage the environment, but you can also choose to manually install the dependencies. Please ensure that the version of Python is >= 3.9.
+1. First, clone this repository and `cd` into it. Then, install the dependencies listed in `requirements.txt`. It is recommended that you use the latest version of Conda to manage the environment, but you can also choose to manually install the dependencies. Please ensure that the version of Python is >= 3.10.
 ```bash
 # Clone the OSWorld repository
 git clone https://github.com/xlang-ai/OSWorld
@@ -52,7 +52,7 @@ git clone https://github.com/xlang-ai/OSWorld
 cd OSWorld
 
 # Optional: Create a Conda environment for OSWorld
-# conda create -n osworld python=3.9
+# conda create -n osworld python=3.10
 # conda activate osworld
 
 # Install required dependencies