Added an evaluation of "Strict YAML" to replace "PyYAML".
Conclusion: not worth the effort
This commit is contained in:
parent
19023ef4e6
commit
734d2b9c35
|
|
@ -0,0 +1,193 @@
|
|||
# Strict YAML
|
||||
|
||||
This is an experiment. The idea was: why not let the YAML parser validate the syntax? Something like a DTD.
|
||||
|
||||
Also: give better information for the error, like position in the file.
|
||||
|
||||
Looking for a solution I found [Strict YAML](https://hitchdev.com/strictyaml/). The main idea is really cool, but the implementation isn't good for users having to write a configuration file.
|
||||
|
||||
## Background
|
||||
|
||||
* I'm comparing with the current [PyYAML](https://pypi.org/project/PyYAML/) parser.
|
||||
* I downloaded the sources from [PyPI](https://pypi.org/project/strictyaml/) v1.0.6.
|
||||
* I then tried the code from [GitHub](https://github.com/crdoconnor/strictyaml) commit 63ceb9ba28e1e6829d0ea597ab8707863f2da1ee
|
||||
* I tried to parse KiPlot's configuration files.
|
||||
* The library depends on [ruamel YAML](https://pypi.org/project/ruamel.yaml/)
|
||||
|
||||
Test code:
|
||||
|
||||
|
||||
```
|
||||
schema_ver = MapPattern(Str(), Any())
|
||||
fname = 'test.yaml'
|
||||
with open(fname) as f:
|
||||
s = f.read()
|
||||
try:
|
||||
parsed = load(s, schema_ver, label=fname)
|
||||
except InconsistentIndentationDisallowed as e:
|
||||
print('Use the same indentation across the file')
|
||||
print(e)
|
||||
sys.exit(1)
|
||||
except YAMLError as e:
|
||||
print('YAML parsing error:')
|
||||
print(e)
|
||||
sys.exit(1)
|
||||
|
||||
schema = Map({"kiplot":
|
||||
Map({"version": Int()}),
|
||||
Optional("preflight"): Map({
|
||||
Optional("run_drc"): Bool(),
|
||||
Optional("run_erc"): Bool(),
|
||||
Optional("update_xml"): Bool(),
|
||||
Optional("check_zone_fills"): Bool(),
|
||||
Optional("ignore_unconnected"): Bool(),
|
||||
}),
|
||||
Optional("outputs"): Seq(Any())})
|
||||
|
||||
try:
|
||||
parsed = load(s, schema, label=fname)
|
||||
except YAMLError as e:
|
||||
print('YAML parsing error:')
|
||||
print(e)
|
||||
sys.exit(1)
|
||||
|
||||
print(repr(parsed))
|
||||
print(parsed['kiplot']['version'])
|
||||
|
||||
```
|
||||
|
||||
|
||||
## What I found
|
||||
|
||||
### Strict indentation forced, can't be disabled
|
||||
|
||||
I understand that detecting inconsistent indentation is a good thing. But looks too strict for my case and can't be disabled:
|
||||
|
||||
```
|
||||
kiplot:
|
||||
version: 1
|
||||
|
||||
preflight:
|
||||
run_drc: true
|
||||
|
||||
```
|
||||
|
||||
Gives:
|
||||
|
||||
```
|
||||
While parsing
|
||||
in "indent.yaml", line 5, column 4:
|
||||
run_drc: true
|
||||
^ (line: 5)
|
||||
Found mapping with indentation inconsistent with previous mapping
|
||||
in "indent.yaml", line 7, column 1:
|
||||
|
||||
^ (line: 7)
|
||||
Nuevamente en el editor
|
||||
|
||||
```
|
||||
|
||||
Here:
|
||||
- The exception can't be masked
|
||||
- The message is hard to understand, line 7 doesn't even exist. The message should point to line 2.
|
||||
- The `InconsistentIndentationDisallowed` is hard to import, needs:
|
||||
|
||||
```
|
||||
from strictyaml.exceptions import InconsistentIndentationDisallowed
|
||||
```
|
||||
|
||||
### Errors seems to be descriptive, but they are missleading
|
||||
|
||||
Schema:
|
||||
|
||||
```
|
||||
schema = Map({"kiplot":
|
||||
Map({"version": Int()}),
|
||||
Optional("preflight"): Map({
|
||||
Optional("run_drc"): Bool(),
|
||||
Optional("run_erc"): Bool(),
|
||||
Optional("update_xml"): Bool(),
|
||||
Optional("check_zone_fills"): Bool(),
|
||||
Optional("ignore_unconnected"): Bool(),
|
||||
}),
|
||||
Optional("outputs"): Seq(Any())})
|
||||
```
|
||||
|
||||
YAML:
|
||||
|
||||
```
|
||||
# Example KiPlot config file
|
||||
kiplot:
|
||||
version: 1
|
||||
|
||||
preflight:
|
||||
run_drc: true
|
||||
run_erc: true
|
||||
update_xml: true
|
||||
check_zone_fills: true
|
||||
ignore_unconnected: false
|
||||
|
||||
outputs:
|
||||
- name: 'gerbers'
|
||||
|
||||
cualquiera:
|
||||
- name: 'gerbers'
|
||||
|
||||
```
|
||||
|
||||
We get:
|
||||
|
||||
```
|
||||
YAML parsing error:
|
||||
while parsing a mapping
|
||||
in "test.yaml", line 14, column 1:
|
||||
|
||||
^ (line: 14)
|
||||
unexpected key not in schema 'cualquiera'
|
||||
in "test.yaml", line 16, column 1:
|
||||
- name: gerbers
|
||||
^ (line: 16)
|
||||
```
|
||||
|
||||
While parsing an empty line? Found 'cualquiera' in line 16?
|
||||
|
||||
It doesn't help more than what we have:
|
||||
|
||||
```
|
||||
Unknown section `cualquiera` in config.
|
||||
```
|
||||
|
||||
### Parser errors are as fuzzy as the ones in PyYAML
|
||||
|
||||
Some errors from ruamel YAML go directly to our application:
|
||||
|
||||
```
|
||||
preflight:
|
||||
check_zone_fills: true
|
||||
ignore_unconnected: false
|
||||
```
|
||||
|
||||
Gives:
|
||||
|
||||
```
|
||||
ruamel.yaml.scanner.ScannerError: mapping values are not allowed here
|
||||
in "test1.yaml", line 3, column 23:
|
||||
ignore_unconnected: false
|
||||
^ (line: 3)
|
||||
```
|
||||
|
||||
This doesn't help more than:
|
||||
|
||||
```
|
||||
mapping values are not allowed here
|
||||
in "scanner_error.yaml", line 3, column 23
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
Using it involves a lot of adaptations in the code and we:
|
||||
|
||||
- Don't gain better messages. They point to lines and columns, but not the right ones.
|
||||
- Need to patch the library in order to make indentation checks optional.
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
kiplot:
|
||||
version: 1
|
||||
|
||||
preflight:
|
||||
run_drc: true
|
||||
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
preflight:
|
||||
check_zone_fills: true
|
||||
ignore_unconnected: false
|
||||
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
#!/usr/bin/python3
|
||||
import os
|
||||
import sys
|
||||
cur_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
# subdir = 'strictyaml-1.0.6'
|
||||
subdir = 'strictyaml'
|
||||
sys.path.append(os.path.join(cur_dir, subdir))
|
||||
# Depende de ruamel: python3-ruamel.yaml
|
||||
# Depende de dateutil: python3-dateutil
|
||||
from strictyaml import (load, Map, Str, Int, Seq, Any, Bool, Optional, MapPattern, YAMLError)
|
||||
from strictyaml.exceptions import InconsistentIndentationDisallowed
|
||||
|
||||
schema_ver = MapPattern(Str(), Any())
|
||||
# fname = 'scanner_error.yaml'
|
||||
# fname = 'indent.yaml'
|
||||
fname = 'test.yaml'
|
||||
with open(fname) as f:
|
||||
s = f.read()
|
||||
try:
|
||||
parsed = load(s, schema_ver, label=fname)
|
||||
except InconsistentIndentationDisallowed as e:
|
||||
print('Use the same indentation across the file')
|
||||
print(e)
|
||||
sys.exit(1)
|
||||
except YAMLError as e:
|
||||
print('YAML parsing error:')
|
||||
print(e)
|
||||
sys.exit(1)
|
||||
|
||||
schema = Map({"kiplot":
|
||||
Map({"version": Int()}),
|
||||
Optional("preflight"): Map({
|
||||
Optional("run_drc"): Bool(),
|
||||
Optional("run_erc"): Bool(),
|
||||
Optional("update_xml"): Bool(),
|
||||
Optional("check_zone_fills"): Bool(),
|
||||
Optional("ignore_unconnected"): Bool(),
|
||||
}),
|
||||
Optional("outputs"): Seq(Any())}) # noqa: E127
|
||||
|
||||
try:
|
||||
parsed = load(s, schema, label=fname)
|
||||
except YAMLError as e:
|
||||
print('YAML parsing error:')
|
||||
print(e)
|
||||
sys.exit(1)
|
||||
|
||||
print(repr(parsed))
|
||||
print(parsed['kiplot']['version'])
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
# Example KiPlot config file
|
||||
kiplot:
|
||||
version: 1
|
||||
|
||||
preflight:
|
||||
run_drc: true
|
||||
run_erc: true
|
||||
update_xml: true
|
||||
check_zone_fills: true
|
||||
ignore_unconnected: false
|
||||
|
||||
outputs:
|
||||
- name: 'gerbers'
|
||||
|
||||
cualquiera:
|
||||
- name: 'gerbers'
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Example KiPlot config file
|
||||
kiplot:
|
||||
version: 1
|
||||
|
||||
preflight:
|
||||
run_drc: true
|
||||
run_erc: true
|
||||
update_xml: true
|
||||
check_zone_fills: true
|
||||
ignore_unconnected: false
|
||||
|
||||
cualquiera:
|
||||
- name: 'gerbers'
|
||||
|
||||
Loading…
Reference in New Issue