86 lines
2.4 KiB
Markdown
86 lines
2.4 KiB
Markdown
<!-- vimc: call SyntaxRange#Include('```xml', '```', 'xml', 'NonText'): -->
|
|
<!-- vimc: call SyntaxRange#Include('```css', '```', 'css', 'NonText'): -->
|
|
<!-- vimc: call SyntaxRange#Include('```sh', '```', 'sh', 'NonText'): -->
|
|
|
|
### About the Converted Accessibility Tree
|
|
|
|
For several applications like Firefox or Thunderbird, you should first enable
|
|
|
|
```sh
|
|
gsettings set org.gnome.desktop.interface toolkit-accessibility true
|
|
```
|
|
|
|
to see their accessibility tree.
|
|
|
|
#### Example of AT
|
|
|
|
An example of a node:
|
|
|
|
```xml
|
|
<section xmlns:attr="uri:deskat:attributes.at-spi.gnome.org" attr:class="subject" st:enabled="true" cp:screencoord="(1525, 169)", cp:windowcoord="(342, 162)", cp:size="(327, 21)">
|
|
歡迎使用新的 Outlook.com 帳戶
|
|
</section>
|
|
```
|
|
|
|
An example of a tree:
|
|
|
|
```xml
|
|
<desktop-frame ...>
|
|
<application name="Thunderbird" ...>
|
|
... <!-- nodes of windows -->
|
|
</application>
|
|
...
|
|
</desktop-frame>
|
|
```
|
|
|
|
#### Useful attributes
|
|
|
|
1. `name` - shows the name of application, title of window, or name of some
|
|
component
|
|
2. `attr:class` - somewhat the same role as `class` in HTML
|
|
3. `attr:id` - somewhat the same role as `id` in HTML
|
|
4. `cp:screencoord` - absolute coordinator on the screen
|
|
5. `cp:windowcoord` - relative coordinator in the window
|
|
6. `cp:size` - the size
|
|
|
|
Also several states like `st:enabled` and `st:visible` can be indicated. A full
|
|
state list is available at
|
|
<https://gitlab.gnome.org/GNOME/pyatspi2/-/blob/master/pyatspi/state.py?ref_type=heads>.
|
|
|
|
#### How to use it in evaluation
|
|
|
|
See example `thunderbird/12086550-11c0-466b-b367-1d9e75b3910e.json` and
|
|
function `check_accessibility_tree` in `metrics/general.py`. You can use CSS
|
|
selector or XPath to reference a target nodes. You can also check its text
|
|
contents.
|
|
|
|
An example of a CSS selector:
|
|
|
|
```css
|
|
application[name=Thunderbird] page-tab-list[attr|id=\"tabmail-tabs\"]>page-tab[name=\"About Profiles\"]
|
|
```
|
|
|
|
This selector will select the page tab of profile manager in Thunderbird (if open).
|
|
|
|
For usage of CSS selector: <https://www.w3.org/TR/selectors-3/>. For usage of XPath: <https://www.w3.org/TR/xpath-31/>.
|
|
|
|
#### Manual check
|
|
|
|
You can use accerciser to check the accessibility tree on GNOME VM.
|
|
|
|
```sh
|
|
sudo apt install accerciser
|
|
```
|
|
|
|
|
|
### Additional Installation
|
|
Activating the window manager control requires the installation of `wmctrl`:
|
|
```bash
|
|
sudo apt install wmctrl
|
|
```
|
|
|
|
To enable recording in the virtual machine, you need to install `ffmpeg`:
|
|
```bash
|
|
sudo apt install ffmpeg
|
|
```
|