diff --git a/README.md b/README.md index 0bcfc72..3222f25 100644 --- a/README.md +++ b/README.md @@ -121,6 +121,18 @@ Please start by reading through the [agent interface](https://github.com/xlang-a Correctly implement the agent interface and import your customized version in the `run.py` file. Afterward, you can execute a command similar to the one in the previous section to run the benchmark on your agent. +## FAQ +### What are the running times and costs under different settings? +| Setting | Expected Time* | Budget Cost (Full Test Set/Small Test Set) | +| ------------------------------ | -------------- | ------------------------------------------ | +| GPT-4V (screenshot) | 10h | $100 ($10) | +| Gemini-ProV (screenshot) | 15h | $0 ($0) | +| Claude-3 Opus (screenshot) | 15h | $150 ($15) | +| GPT-4V (a11y tree, SoM, etc.) | 30h | $500 ($50) | + +\*No environment parallelism. Calculated in April 2024. + + ## Citation If you find this environment useful, please consider citing our work: ```