Added an evaluation of "Strict YAML" to replace "PyYAML".

Conclusion: not worth the effort
This commit is contained in:
Salvador E. Tropea 2020-06-21 11:57:03 -03:00
parent 19023ef4e6
commit 734d2b9c35
6 changed files with 283 additions and 0 deletions

View File

@ -0,0 +1,193 @@
# Strict YAML
This is an experiment. The idea was: why not let the YAML parser validate the syntax? Something like a DTD.
Also: give better information for the error, like position in the file.
Looking for a solution I found [Strict YAML](https://hitchdev.com/strictyaml/). The main idea is really cool, but the implementation isn't good for users having to write a configuration file.
## Background
* I'm comparing with the current [PyYAML](https://pypi.org/project/PyYAML/) parser.
* I downloaded the sources from [PyPI](https://pypi.org/project/strictyaml/) v1.0.6.
* I then tried the code from [GitHub](https://github.com/crdoconnor/strictyaml) commit 63ceb9ba28e1e6829d0ea597ab8707863f2da1ee
* I tried to parse KiPlot's configuration files.
* The library depends on [ruamel YAML](https://pypi.org/project/ruamel.yaml/)
Test code:
```
schema_ver = MapPattern(Str(), Any())
fname = 'test.yaml'
with open(fname) as f:
s = f.read()
try:
parsed = load(s, schema_ver, label=fname)
except InconsistentIndentationDisallowed as e:
print('Use the same indentation across the file')
print(e)
sys.exit(1)
except YAMLError as e:
print('YAML parsing error:')
print(e)
sys.exit(1)
schema = Map({"kiplot":
Map({"version": Int()}),
Optional("preflight"): Map({
Optional("run_drc"): Bool(),
Optional("run_erc"): Bool(),
Optional("update_xml"): Bool(),
Optional("check_zone_fills"): Bool(),
Optional("ignore_unconnected"): Bool(),
}),
Optional("outputs"): Seq(Any())})
try:
parsed = load(s, schema, label=fname)
except YAMLError as e:
print('YAML parsing error:')
print(e)
sys.exit(1)
print(repr(parsed))
print(parsed['kiplot']['version'])
```
## What I found
### Strict indentation forced, can't be disabled
I understand that detecting inconsistent indentation is a good thing. But looks too strict for my case and can't be disabled:
```
kiplot:
version: 1
preflight:
run_drc: true
```
Gives:
```
While parsing
in "indent.yaml", line 5, column 4:
run_drc: true
^ (line: 5)
Found mapping with indentation inconsistent with previous mapping
in "indent.yaml", line 7, column 1:
^ (line: 7)
Nuevamente en el editor
```
Here:
- The exception can't be masked
- The message is hard to understand, line 7 doesn't even exist. The message should point to line 2.
- The `InconsistentIndentationDisallowed` is hard to import, needs:
```
from strictyaml.exceptions import InconsistentIndentationDisallowed
```
### Errors seems to be descriptive, but they are missleading
Schema:
```
schema = Map({"kiplot":
Map({"version": Int()}),
Optional("preflight"): Map({
Optional("run_drc"): Bool(),
Optional("run_erc"): Bool(),
Optional("update_xml"): Bool(),
Optional("check_zone_fills"): Bool(),
Optional("ignore_unconnected"): Bool(),
}),
Optional("outputs"): Seq(Any())})
```
YAML:
```
# Example KiPlot config file
kiplot:
version: 1
preflight:
run_drc: true
run_erc: true
update_xml: true
check_zone_fills: true
ignore_unconnected: false
outputs:
- name: 'gerbers'
cualquiera:
- name: 'gerbers'
```
We get:
```
YAML parsing error:
while parsing a mapping
in "test.yaml", line 14, column 1:
^ (line: 14)
unexpected key not in schema 'cualquiera'
in "test.yaml", line 16, column 1:
- name: gerbers
^ (line: 16)
```
While parsing an empty line? Found 'cualquiera' in line 16?
It doesn't help more than what we have:
```
Unknown section `cualquiera` in config.
```
### Parser errors are as fuzzy as the ones in PyYAML
Some errors from ruamel YAML go directly to our application:
```
preflight:
check_zone_fills: true
ignore_unconnected: false
```
Gives:
```
ruamel.yaml.scanner.ScannerError: mapping values are not allowed here
in "test1.yaml", line 3, column 23:
ignore_unconnected: false
^ (line: 3)
```
This doesn't help more than:
```
mapping values are not allowed here
in "scanner_error.yaml", line 3, column 23
```
## Conclusion
Using it involves a lot of adaptations in the code and we:
- Don't gain better messages. They point to lines and columns, but not the right ones.
- Need to patch the library in order to make indentation checks optional.

View File

@ -0,0 +1,6 @@
kiplot:
version: 1
preflight:
run_drc: true

View File

@ -0,0 +1,4 @@
preflight:
check_zone_fills: true
ignore_unconnected: false

View File

@ -0,0 +1,49 @@
#!/usr/bin/python3
import os
import sys
cur_dir = os.path.dirname(os.path.abspath(__file__))
# subdir = 'strictyaml-1.0.6'
subdir = 'strictyaml'
sys.path.append(os.path.join(cur_dir, subdir))
# Depende de ruamel: python3-ruamel.yaml
# Depende de dateutil: python3-dateutil
from strictyaml import (load, Map, Str, Int, Seq, Any, Bool, Optional, MapPattern, YAMLError)
from strictyaml.exceptions import InconsistentIndentationDisallowed
schema_ver = MapPattern(Str(), Any())
# fname = 'scanner_error.yaml'
# fname = 'indent.yaml'
fname = 'test.yaml'
with open(fname) as f:
s = f.read()
try:
parsed = load(s, schema_ver, label=fname)
except InconsistentIndentationDisallowed as e:
print('Use the same indentation across the file')
print(e)
sys.exit(1)
except YAMLError as e:
print('YAML parsing error:')
print(e)
sys.exit(1)
schema = Map({"kiplot":
Map({"version": Int()}),
Optional("preflight"): Map({
Optional("run_drc"): Bool(),
Optional("run_erc"): Bool(),
Optional("update_xml"): Bool(),
Optional("check_zone_fills"): Bool(),
Optional("ignore_unconnected"): Bool(),
}),
Optional("outputs"): Seq(Any())}) # noqa: E127
try:
parsed = load(s, schema, label=fname)
except YAMLError as e:
print('YAML parsing error:')
print(e)
sys.exit(1)
print(repr(parsed))
print(parsed['kiplot']['version'])

View File

@ -0,0 +1,17 @@
# Example KiPlot config file
kiplot:
version: 1
preflight:
run_drc: true
run_erc: true
update_xml: true
check_zone_fills: true
ignore_unconnected: false
outputs:
- name: 'gerbers'
cualquiera:
- name: 'gerbers'

View File

@ -0,0 +1,14 @@
# Example KiPlot config file
kiplot:
version: 1
preflight:
run_drc: true
run_erc: true
update_xml: true
check_zone_fills: true
ignore_unconnected: false
cualquiera:
- name: 'gerbers'