feat: enhance run_coact.py and related agents with improved task handling and configuration

- Updated TASK_DESCRIPTION in run_coact.py to clarify task-solving steps and requirements.
- Modified configuration parameters for provider name and client password for better security and flexibility.
- Enhanced OrchestratorUserProxyAgent to include user instruction in the auto-reply and improved screenshot handling.
- Adjusted coding_agent.py to ensure proper verification of results before saving changes.
- Improved CUA agent prompts to maintain application state and handle user instructions more effectively.
- Ensured existing code logic remains unchanged while enhancing functionality and usability.
This commit is contained in:
Timothyxxx
2025-08-13 09:04:09 +00:00
parent d2ae0f697d
commit 7fb5860da0
4 changed files with 100 additions and 59 deletions

View File

@@ -7,22 +7,20 @@ from .autogen.agentchat.contrib.multimodal_conversable_agent import MultimodalCo
CODER_SYSTEM_MESSAGE = """# Your role
You are a coding assistant, you need to solve a task step-by-step given by the user.
You can write code in ```bash...``` code blocks for bash scripts, and ```python...``` code blocks for python code.
# Important notes
- Once you complete the task, reply ONLY with "TERMINATE" to end the task.
- DO NOT mix the TERMINATE with any other words or code blocks in your reply.
- When you write code, you must identify the language (whether it is python or bash) of the code.
- You are a programmer, you need to solve a task step-by-step given by the user.
- You can write code in ```bash...``` code blocks for bash scripts, and ```python...``` code blocks for python code.
- Your linux username is "user".
- Wrap all your code in ONE code block. DO NOT let user save the code as a file and execute it for you.
- If you want to use sudo, follow the format: "echo {CLIENT_PASSWORD} | sudo -S [YOUR COMMANDS]" (no quotes for the word "{CLIENT_PASSWORD}").
- Ignore the error: "sudo: /etc/sudoers.d is world writable".
- Your python code will be sent line-by-line into a interactive python terminal. Do not include __main__ in your code.
- When import a package, you need to check if the package is installed. If not, you need to install it yourself.
# Requirements
- You MUST verify the result before save the changes.
- When you write code, you must identify the language (whether it is python or bash) of the code.
- Wrap all your code in ONE code block. DO NOT let user save the code as a file and execute it for you.
- Do not include __main__ in your python code.
- When you modify a spreadsheet, **make sure every value is in the expected cell**.
- When importing a package, you need to check if the package has been installed. If not, you need to install it yourself.
- You need to print the progressive and final result.
- If you met execution error, you need to analyze the error message and try to fix the error.
- IMPORTANT: If you modified a file like spreadsheet, you should close and reopen the file by operating the GUI, so that I can see what you changed.
"""
class TerminalProxyAgent(MultimodalConversableAgent):
@@ -62,15 +60,15 @@ class TerminalProxyAgent(MultimodalConversableAgent):
exitcode = 0
logs = output_dict["output"]
else:
exitcode = -1
exitcode = 0
logs = output_dict["output"]
elif lang in PYTHON_VARIANTS:
output_dict = self.env.controller.run_python_script(code)
if output_dict["status"] == "error":
exitcode = -1
exitcode = 0
logs = output_dict["output"]
else:
exitcode = -1
exitcode = 0
logs = output_dict["message"]
else:
exitcode = -1