lzy/sci-gui-agent-benchmark
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ver Jan23rd

Browse Source 
updated README for Calc evaluation
This commit is contained in:
David Chang
2024-01-23 22:53:58 +08:00
parent 99d3712bfb
commit 2f8e1b6a50
1 changed files with 32 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
desktop_env/evaluators/README.md
View File

@@ -24,9 +24,39 @@ xmltodict
libreoffice --convert-to "csv:Text - txt - csv (StarCalc):44,34,UTF8,,,,false,true,true,false,false,1" --out-dir /home/user /home/user/abc.xlsx
```
This command will generate `abc-Sheet1.csv` under `/home/user`. The last `1` in the conversion options indicates the sheet number (starting from 1) to export. Detailed usage should be referred to at [CSV Filter Options](https://help.libreoffice.org/latest/ro/text/shared/guide/csv_params.html).
This command will generate `abc-Sheet1.csv` under `/home/user`. The last `1` in
the conversion options indicates the sheet number (starting from 1) to export.
Detailed usage should be referred to at [CSV Filter
Options](https://help.libreoffice.org/latest/ro/text/shared/guide/csv_params.html).
Refer to `libreoffice_calc/21df9241-f8d7-4509-b7f1-37e501a823f7.json` for an example.
Refer to `libreoffice_calc/21df9241-f8d7-4509-b7f1-37e501a823f7.json` for an
example.
### About `compare_table`
Evaluation to xlsx files mainly relies on `compare_table`. It accepts two file
names and a list of rules defined as `options`. Refer to
`libreoffice_calc/21df9241-f8d7-4509-b7f1-37e501a823f7.json` for an example.
In each rule, there is a required field `type`. The supported types are defined
in `compare_table` function. The most common two are `sheet_data` and
`sheet_print`. `sheet_data` compares the internal cell values through pandoc,
while `sheet_print` compares the shown cell values through csv. A csv should be
generated and downloaded for `sheet_print`. See the previous section and
example in `libreoffice_calc/21df9241-f8d7-4509-b7f1-37e501a823f7.json`.
Other fields in a rule are described for each evaluation type in
`compare_table` function. `sheet_idx0` (or `sheet_idx1`, `sheet_idx`) is a
common field to indicate which sheet is to extracted from the workbook. If an
integer i is given, then it extracts the i-th sheet from result xlsx (i starts
from 0). If a string is given, it should be preceded with "RI", "RN", "EI", or
"EN". "R" indicates to extract from result xlsx while "E" indicates to extract
from expected (golden) xlsx. "I" indicates a sheet number (starting from 0) and
"N" indicates a sheet name (usually, they're like "Sheet1", "Sheet2", ...).
Some rules use a atructure like `{"method": "eq", "ref": "abc"}`. These rules
are checked through `utils._match_value_to_rule` function. Check it for the
implemented matching methods.
## Chrome