Update getters for Chrome and VLC media player software
This commit is contained in:
Timothyxxx
132
desktop_env/evaluators/metrics/README.md
Normal file
132
desktop_env/evaluators/metrics/README.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# Setup Instructions
|
||||
|
||||
|
||||
## Chrome
|
||||
|
||||
### Starting Chrome with Remote Debugging for Python
|
||||
|
||||
To enable remote debugging in Chrome, which allows tools like Playwright for Python to connect to and control an existing Chrome instance, follow these steps:
|
||||
|
||||
#### Manually Enabling Remote Debugging in Chrome
|
||||
|
||||
1. **Locate the Chrome Shortcut**:
|
||||
- Find the Chrome shortcut that you usually use to open the browser. This could be on your desktop, start menu, or taskbar.
|
||||
|
||||
2. **Edit Shortcut Properties**:
|
||||
- Right-click on the Chrome shortcut and select `Properties`.
|
||||
|
||||
3. **Modify the Target Field**:
|
||||
- In the `Target` field, add `--remote-debugging-port=9222` at the end of the path. Ensure there is a space between the path and the flag you add.
|
||||
- It should look something like this: `"C:\Path\To\Chrome.exe" --remote-debugging-port=9222`.
|
||||
|
||||
4. **Apply and Close**:
|
||||
- Click `Apply` and then `OK` to close the dialog.
|
||||
|
||||
5. **Start Chrome**:
|
||||
- Use this modified shortcut to start Chrome. Chrome will now start with remote debugging enabled on port 9222.
|
||||
|
||||
6. **Confirm Remote Debugging**:
|
||||
- Open a browser and navigate to `http://localhost:9222`. If you see a webpage with information about active tabs, remote debugging is working.
|
||||
|
||||
---
|
||||
|
||||
### Setting Up Playwright for Python
|
||||
|
||||
Playwright for Python is a browser automation library to control Chromium, Firefox, and WebKit with a single API.
|
||||
|
||||
#### Installing Playwright
|
||||
|
||||
- Ensure you have Python installed on your system. If not, download and install it from the [Python official website](https://www.python.org/).
|
||||
|
||||
- Install Playwright using pip (Python Package Installer). Open a command line or terminal and run:
|
||||
|
||||
```bash
|
||||
pip install playwright
|
||||
```
|
||||
|
||||
- After installing Playwright, you need to run the install command to download the necessary browser binaries:
|
||||
|
||||
```bash
|
||||
playwright install
|
||||
```
|
||||
|
||||
#### Writing a Playwright Script in Python
|
||||
|
||||
- Create a Python file for your automation script.
|
||||
|
||||
- Import the Playwright module at the beginning of your script:
|
||||
|
||||
```python
|
||||
from playwright.sync_api import sync_playwright
|
||||
```
|
||||
|
||||
- You can now use Playwright's API to control browsers.
|
||||
|
||||
#### Example Playwright Script
|
||||
|
||||
Here is a simple example to open a page using Playwright:
|
||||
|
||||
```python
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
def run(playwright):
|
||||
browser = playwright.chromium.launch()
|
||||
page = browser.new_page()
|
||||
page.goto("http://example.com")
|
||||
## other actions...
|
||||
browser.close()
|
||||
|
||||
with sync_playwright() as playwright:
|
||||
run(playwright)
|
||||
```
|
||||
|
||||
- This script launches Chromium, opens a new page, navigates to `example.com`, and then closes the browser.
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
- If you encounter issues with Playwright, ensure that your Python environment is correctly set up and that you have installed Playwright and its dependencies correctly.
|
||||
- For detailed documentation, visit the [Playwright for Python Documentation](https://playwright.dev/python/docs/intro).
|
||||
|
||||
|
||||
## VLC Media Player
|
||||
### Setting Up VLC's HTTP Interface
|
||||
|
||||
To enable and use the HTTP interface in VLC Media Player for remote control and status checks, follow these steps:
|
||||
|
||||
#### 1. Open VLC Preferences
|
||||
|
||||
- Open VLC Media Player.
|
||||
- Go to `Tools` > `Preferences` from the menu.
|
||||
|
||||
#### 2. Show All Settings
|
||||
|
||||
- In the Preferences window, at the bottom left corner, select `All` under `Show settings` to display advanced settings.
|
||||
|
||||
#### 3. Enable Main Interfaces
|
||||
|
||||
- In the advanced preferences, expand the `Interface` section.
|
||||
- Click on `Main interfaces`.
|
||||
- Check the box for `Web` to enable the HTTP interface.
|
||||
|
||||
#### 4. Configure Lua HTTP
|
||||
|
||||
- Expand the `Main interfaces` node and select `Lua`.
|
||||
- Under `Lua HTTP`, set a password in the `Lua HTTP` section. This password will be required to access the HTTP interface.
|
||||
|
||||
#### 5. Save and Restart VLC
|
||||
|
||||
- Click `Save` to apply the changes.
|
||||
- Restart VLC Media Player for the changes to take effect.
|
||||
|
||||
#### 6. Accessing the HTTP Interface
|
||||
|
||||
- Open a web browser and go to `http://localhost:8080`.
|
||||
- You will be prompted for a password. Enter the password you set in the Lua HTTP settings.
|
||||
- Once logged in, you will have access to VLC's HTTP interface for remote control.
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
- If you cannot access the HTTP interface, check if your firewall or security software is blocking the connection.
|
||||
- Ensure VLC is running and the correct port (default is 8080) is being used.
|
||||
- If the port is in use by another application, you may change the port number in VLC's settings.
|
||||
|
||||
@@ -3,6 +3,8 @@ import os
|
||||
import platform
|
||||
import sqlite3
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
"""
|
||||
WARNING:
|
||||
1. Functions from this script assume that no account is registered on Chrome, otherwise the default file path needs to be changed.
|
||||
@@ -12,6 +14,7 @@ WARNING:
|
||||
|
||||
# todo: move to getter module
|
||||
|
||||
# The following ones just need to load info from the files of software, no need to connect to the software
|
||||
def get_default_search_engine():
|
||||
if platform.system() == 'Windows':
|
||||
preference_file_path = os.path.join(os.getenv('LOCALAPPDATA'),
|
||||
@@ -19,8 +22,10 @@ def get_default_search_engine():
|
||||
elif platform.system() == 'Darwin':
|
||||
preference_file_path = os.path.join(os.getenv('HOME'),
|
||||
'Library/Application Support/Google/Chrome/Default/Preferences')
|
||||
else:
|
||||
elif platform.system() == 'Linux':
|
||||
preference_file_path = os.path.join(os.getenv('HOME'), '.config/google-chrome/Default/Preferences')
|
||||
else:
|
||||
raise Exception('Unsupported operating system')
|
||||
|
||||
try:
|
||||
with open(preference_file_path, 'r', encoding='utf-8') as file:
|
||||
@@ -41,8 +46,10 @@ def get_cookie_data():
|
||||
elif platform.system() == 'Darwin':
|
||||
chrome_cookie_file_path = os.path.join(os.getenv('HOME'),
|
||||
'Library/Application Support/Google/Chrome/Default/Cookies')
|
||||
else:
|
||||
elif platform.system() == 'Linux':
|
||||
chrome_cookie_file_path = os.path.join(os.getenv('HOME'), '.config/google-chrome/Default/Cookies')
|
||||
else:
|
||||
raise Exception('Unsupported operating system')
|
||||
|
||||
try:
|
||||
conn = sqlite3.connect(chrome_cookie_file_path)
|
||||
@@ -65,8 +72,10 @@ def get_bookmarks():
|
||||
elif platform.system() == 'Darwin':
|
||||
preference_file_path = os.path.join(os.getenv('HOME'),
|
||||
'Library/Application Support/Google/Chrome/Default/Bookmarks')
|
||||
else:
|
||||
elif platform.system() == 'Linux':
|
||||
preference_file_path = os.path.join(os.getenv('HOME'), '.config/google-chrome/Default/Bookmarks')
|
||||
else:
|
||||
raise Exception('Unsupported operating system')
|
||||
|
||||
try:
|
||||
with open(preference_file_path, 'r', encoding='utf-8') as file:
|
||||
@@ -78,3 +87,75 @@ def get_bookmarks():
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def get_extensions_installed_from_shop():
|
||||
"""Find the Chrome extensions directory based on the operating system."""
|
||||
os_name = platform.system()
|
||||
if os_name == 'Windows':
|
||||
chrome_extension_dir = os.path.expanduser(
|
||||
'~') + '\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Extensions\\'
|
||||
elif os_name == 'Darwin': # macOS
|
||||
chrome_extension_dir = os.path.expanduser(
|
||||
'~') + '/Library/Application Support/Google/Chrome/Default/Extensions/'
|
||||
elif os_name == 'Linux':
|
||||
chrome_extension_dir = os.path.expanduser('~') + '/.config/google-chrome/Default/Extensions/'
|
||||
else:
|
||||
raise Exception('Unsupported operating system')
|
||||
|
||||
manifests = []
|
||||
for extension_id in os.listdir(chrome_extension_dir):
|
||||
extension_path = os.path.join(chrome_extension_dir, extension_id)
|
||||
if os.path.isdir(extension_path):
|
||||
# Iterate through version-named subdirectories
|
||||
for version_dir in os.listdir(extension_path):
|
||||
version_path = os.path.join(extension_path, version_dir)
|
||||
manifest_path = os.path.join(version_path, 'manifest.json')
|
||||
if os.path.isfile(manifest_path):
|
||||
with open(manifest_path, 'r') as file:
|
||||
try:
|
||||
manifest = json.load(file)
|
||||
manifests.append(manifest)
|
||||
except json.JSONDecodeError:
|
||||
print(f"Error reading {manifest_path}")
|
||||
return manifests
|
||||
|
||||
|
||||
# The following ones require Playwright to be installed on the target machine, and the chrome needs to be pre-config on port info to allow remote debugging, see README.md for details
|
||||
|
||||
def get_open_tabs_info(remote_debugging_url):
|
||||
with sync_playwright() as p:
|
||||
# connect to remote Chrome instance
|
||||
browser = p.chromium.connect_over_cdp(remote_debugging_url)
|
||||
|
||||
tabs_info = []
|
||||
for context in browser.contexts:
|
||||
for page in context.pages:
|
||||
title = page.title()
|
||||
url = page.url
|
||||
tabs_info.append({'title': title, 'url': url})
|
||||
|
||||
browser.close()
|
||||
return tabs_info
|
||||
|
||||
|
||||
def get_active_tab_info(remote_debugging_url):
|
||||
with sync_playwright() as p:
|
||||
# connect to remote Chrome instance
|
||||
browser = p.chromium.connect_over_cdp(remote_debugging_url)
|
||||
|
||||
active_tab_info = {}
|
||||
for context in browser.contexts:
|
||||
for page in context.pages():
|
||||
if page.is_visible("body"): # check the visibility of the page body to determine the active status
|
||||
active_tab_info = {
|
||||
'title': page.title(),
|
||||
'url': page.url,
|
||||
'content': page.content() # get the HTML content of the page
|
||||
}
|
||||
break
|
||||
if active_tab_info:
|
||||
break
|
||||
|
||||
browser.close()
|
||||
return active_tab_info
|
||||
|
||||
Reference in New Issue
Block a user