Update getters for Chrome and VLC media player software

desktop_env/evaluators/metrics/README.md

@@ -0,0 +1,132 @@
+# Setup Instructions
+
+
+## Chrome
+
+### Starting Chrome with Remote Debugging for Python
+
+To enable remote debugging in Chrome, which allows tools like Playwright for Python to connect to and control an existing Chrome instance, follow these steps:
+
+#### Manually Enabling Remote Debugging in Chrome
+
+1. **Locate the Chrome Shortcut**:
+   - Find the Chrome shortcut that you usually use to open the browser. This could be on your desktop, start menu, or taskbar.
+
+2. **Edit Shortcut Properties**:
+   - Right-click on the Chrome shortcut and select `Properties`.
+
+3. **Modify the Target Field**:
+   - In the `Target` field, add `--remote-debugging-port=9222` at the end of the path. Ensure there is a space between the path and the flag you add.
+   - It should look something like this: `"C:\Path\To\Chrome.exe" --remote-debugging-port=9222`.
+
+4. **Apply and Close**:
+   - Click `Apply` and then `OK` to close the dialog.
+
+5. **Start Chrome**:
+   - Use this modified shortcut to start Chrome. Chrome will now start with remote debugging enabled on port 9222.
+
+6. **Confirm Remote Debugging**:
+   - Open a browser and navigate to `http://localhost:9222`. If you see a webpage with information about active tabs, remote debugging is working.
+
+---
+
+### Setting Up Playwright for Python
+
+Playwright for Python is a browser automation library to control Chromium, Firefox, and WebKit with a single API.
+
+#### Installing Playwright
+
+- Ensure you have Python installed on your system. If not, download and install it from the [Python official website](https://www.python.org/).
+
+- Install Playwright using pip (Python Package Installer). Open a command line or terminal and run:
+
+  ```bash
+  pip install playwright
+  ```
+
+- After installing Playwright, you need to run the install command to download the necessary browser binaries:
+
+  ```bash
+  playwright install
+  ```
+
+#### Writing a Playwright Script in Python
+
+- Create a Python file for your automation script.
+
+- Import the Playwright module at the beginning of your script:
+
+  ```python
+  from playwright.sync_api import sync_playwright
+  ```
+
+- You can now use Playwright's API to control browsers.
+
+#### Example Playwright Script
+
+Here is a simple example to open a page using Playwright:
+
+```python
+from playwright.sync_api import sync_playwright
+
+def run(playwright):
+    browser = playwright.chromium.launch()
+    page = browser.new_page()
+    page.goto("http://example.com")
+    ## other actions...
+    browser.close()
+
+with sync_playwright() as playwright:
+    run(playwright)
+```
+
+- This script launches Chromium, opens a new page, navigates to `example.com`, and then closes the browser.
+
+#### Troubleshooting
+
+- If you encounter issues with Playwright, ensure that your Python environment is correctly set up and that you have installed Playwright and its dependencies correctly.
+- For detailed documentation, visit the [Playwright for Python Documentation](https://playwright.dev/python/docs/intro).
+
+
+## VLC Media Player
+### Setting Up VLC's HTTP Interface
+
+To enable and use the HTTP interface in VLC Media Player for remote control and status checks, follow these steps:
+
+#### 1. Open VLC Preferences
+
+- Open VLC Media Player.
+- Go to `Tools` > `Preferences` from the menu.
+
+#### 2. Show All Settings
+
+- In the Preferences window, at the bottom left corner, select `All` under `Show settings` to display advanced settings.
+
+#### 3. Enable Main Interfaces
+
+- In the advanced preferences, expand the `Interface` section.
+- Click on `Main interfaces`.
+- Check the box for `Web` to enable the HTTP interface.
+
+#### 4. Configure Lua HTTP
+
+- Expand the `Main interfaces` node and select `Lua`.
+- Under `Lua HTTP`, set a password in the `Lua HTTP` section. This password will be required to access the HTTP interface.
+
+#### 5. Save and Restart VLC
+
+- Click `Save` to apply the changes.
+- Restart VLC Media Player for the changes to take effect.
+
+#### 6. Accessing the HTTP Interface
+
+- Open a web browser and go to `http://localhost:8080`.
+- You will be prompted for a password. Enter the password you set in the Lua HTTP settings.
+- Once logged in, you will have access to VLC's HTTP interface for remote control.
+
+#### Troubleshooting
+
+- If you cannot access the HTTP interface, check if your firewall or security software is blocking the connection.
+- Ensure VLC is running and the correct port (default is 8080) is being used.
+- If the port is in use by another application, you may change the port number in VLC's settings.
+

desktop_env/evaluators/metrics/chrome.py

@@ -3,6 +3,8 @@ import os
 import platform
 import sqlite3
 
+from playwright.sync_api import sync_playwright
+
 """
 WARNING: 
 1. Functions from this script assume that no account is registered on Chrome, otherwise the default file path needs to be changed.
@@ -12,6 +14,7 @@ WARNING:
 
 # todo: move to getter module
 
+# The following ones just need to load info from the files of software, no need to connect to the software
 def get_default_search_engine():
     if platform.system() == 'Windows':
         preference_file_path = os.path.join(os.getenv('LOCALAPPDATA'),
@@ -19,8 +22,10 @@ def get_default_search_engine():
     elif platform.system() == 'Darwin':
         preference_file_path = os.path.join(os.getenv('HOME'),
                                             'Library/Application Support/Google/Chrome/Default/Preferences')
-    else:
+    elif platform.system() == 'Linux':
         preference_file_path = os.path.join(os.getenv('HOME'), '.config/google-chrome/Default/Preferences')
+    else:
+        raise Exception('Unsupported operating system')
 
     try:
         with open(preference_file_path, 'r', encoding='utf-8') as file:
@@ -41,8 +46,10 @@ def get_cookie_data():
     elif platform.system() == 'Darwin':
         chrome_cookie_file_path = os.path.join(os.getenv('HOME'),
                                                'Library/Application Support/Google/Chrome/Default/Cookies')
-    else:
+    elif platform.system() == 'Linux':
         chrome_cookie_file_path = os.path.join(os.getenv('HOME'), '.config/google-chrome/Default/Cookies')
+    else:
+        raise Exception('Unsupported operating system')
 
     try:
         conn = sqlite3.connect(chrome_cookie_file_path)
@@ -65,8 +72,10 @@ def get_bookmarks():
     elif platform.system() == 'Darwin':
         preference_file_path = os.path.join(os.getenv('HOME'),
                                             'Library/Application Support/Google/Chrome/Default/Bookmarks')
-    else:
+    elif platform.system() == 'Linux':
         preference_file_path = os.path.join(os.getenv('HOME'), '.config/google-chrome/Default/Bookmarks')
+    else:
+        raise Exception('Unsupported operating system')
 
     try:
         with open(preference_file_path, 'r', encoding='utf-8') as file:
@@ -78,3 +87,75 @@ def get_bookmarks():
     except Exception as e:
         print(f"Error: {e}")
         return None
+
+
+def get_extensions_installed_from_shop():
+    """Find the Chrome extensions directory based on the operating system."""
+    os_name = platform.system()
+    if os_name == 'Windows':
+        chrome_extension_dir = os.path.expanduser(
+            '~') + '\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Extensions\\'
+    elif os_name == 'Darwin':  # macOS
+        chrome_extension_dir = os.path.expanduser(
+            '~') + '/Library/Application Support/Google/Chrome/Default/Extensions/'
+    elif os_name == 'Linux':
+        chrome_extension_dir = os.path.expanduser('~') + '/.config/google-chrome/Default/Extensions/'
+    else:
+        raise Exception('Unsupported operating system')
+
+    manifests = []
+    for extension_id in os.listdir(chrome_extension_dir):
+        extension_path = os.path.join(chrome_extension_dir, extension_id)
+        if os.path.isdir(extension_path):
+            # Iterate through version-named subdirectories
+            for version_dir in os.listdir(extension_path):
+                version_path = os.path.join(extension_path, version_dir)
+                manifest_path = os.path.join(version_path, 'manifest.json')
+                if os.path.isfile(manifest_path):
+                    with open(manifest_path, 'r') as file:
+                        try:
+                            manifest = json.load(file)
+                            manifests.append(manifest)
+                        except json.JSONDecodeError:
+                            print(f"Error reading {manifest_path}")
+    return manifests
+
+
+# The following ones require Playwright to be installed on the target machine, and the chrome needs to be pre-config on port info to allow remote debugging, see README.md for details
+
+def get_open_tabs_info(remote_debugging_url):
+    with sync_playwright() as p:
+        # connect to remote Chrome instance
+        browser = p.chromium.connect_over_cdp(remote_debugging_url)
+
+        tabs_info = []
+        for context in browser.contexts:
+            for page in context.pages:
+                title = page.title()
+                url = page.url
+                tabs_info.append({'title': title, 'url': url})
+
+        browser.close()
+        return tabs_info
+
+
+def get_active_tab_info(remote_debugging_url):
+    with sync_playwright() as p:
+        # connect to remote Chrome instance
+        browser = p.chromium.connect_over_cdp(remote_debugging_url)
+
+        active_tab_info = {}
+        for context in browser.contexts:
+            for page in context.pages():
+                if page.is_visible("body"):  # check the visibility of the page body to determine the active status
+                    active_tab_info = {
+                        'title': page.title(),
+                        'url': page.url,
+                        'content': page.content()  # get the HTML content of the page
+                    }
+                    break
+            if active_tab_info:
+                break
+
+        browser.close()
+        return active_tab_info