KiBot/experiments/strict_yaml

Strict YAML

This is an experiment. The idea was: why not let the YAML parser validate the syntax? Something like a DTD.

Also: give better information for the error, like position in the file.

Looking for a solution I found Strict YAML. The main idea is really cool, but the implementation isn't good for users having to write a configuration file.

Background

• I'm comparing with the current PyYAML parser.
• I downloaded the sources from PyPI v1.0.6.
• I then tried the code from GitHub commit 63ceb9ba28e1e6829d0ea597ab8707863f2da1ee
• I tried to parse KiPlot's configuration files.
• The library depends on ruamel YAML

Test code:

schema_ver = MapPattern(Str(), Any())
fname = 'test.yaml'
with open(fname) as f:
    s = f.read()
try:
    parsed = load(s, schema_ver, label=fname)
except InconsistentIndentationDisallowed as e:
    print('Use the same indentation across the file')
    print(e)
    sys.exit(1)
except YAMLError as e:
    print('YAML parsing error:')
    print(e)
    sys.exit(1)

schema = Map({"kiplot":
                 Map({"version": Int()}),
              Optional("preflight"): Map({
                 Optional("run_drc"): Bool(),
                 Optional("run_erc"): Bool(),
                 Optional("update_xml"): Bool(),
                 Optional("check_zone_fills"): Bool(),
                 Optional("ignore_unconnected"): Bool(),
              }),
              Optional("outputs"): Seq(Any())})

try:
    parsed = load(s, schema, label=fname)
except YAMLError as e:
    print('YAML parsing error:')
    print(e)
    sys.exit(1)

print(repr(parsed))
print(parsed['kiplot']['version'])

What I found

Strict indentation forced, can't be disabled

I understand that detecting inconsistent indentation is a good thing. But looks too strict for my case and can't be disabled:

kiplot:
  version: 1

preflight:
   run_drc: true

Gives:

While parsing
  in "indent.yaml", line 5, column 4:
       run_drc: true
       ^ (line: 5)
Found mapping with indentation inconsistent with previous mapping
  in "indent.yaml", line 7, column 1:
    
    ^ (line: 7)
Nuevamente en el editor

Here:

• The exception can't be masked
• The message is hard to understand, line 7 doesn't even exist. The message should point to line 2.
• The InconsistentIndentationDisallowed is hard to import, needs:

from strictyaml.exceptions import InconsistentIndentationDisallowed

Errors seems to be descriptive, but they are missleading

Schema:

schema = Map({"kiplot":
                 Map({"version": Int()}),
              Optional("preflight"): Map({
                 Optional("run_drc"): Bool(),
                 Optional("run_erc"): Bool(),
                 Optional("update_xml"): Bool(),
                 Optional("check_zone_fills"): Bool(),
                 Optional("ignore_unconnected"): Bool(),
              }),
              Optional("outputs"): Seq(Any())})

YAML:

# Example KiPlot config file
kiplot:
   version: 1

preflight:
   run_drc: true
   run_erc: true
   update_xml: true
   check_zone_fills: true
   ignore_unconnected: false

outputs:
  - name: 'gerbers'

cualquiera:
  - name: 'gerbers'

We get:

YAML parsing error:
while parsing a mapping
  in "test.yaml", line 14, column 1:
    
    ^ (line: 14)
unexpected key not in schema 'cualquiera'
  in "test.yaml", line 16, column 1:
    - name: gerbers
    ^ (line: 16)

While parsing an empty line? Found 'cualquiera' in line 16?

It doesn't help more than what we have:

Unknown section `cualquiera` in config.

Parser errors are as fuzzy as the ones in PyYAML

Some errors from ruamel YAML go directly to our application:

preflight:
   check_zone_fills: true
    ignore_unconnected: false

Gives:

ruamel.yaml.scanner.ScannerError: mapping values are not allowed here
  in "test1.yaml", line 3, column 23:
        ignore_unconnected: false
                          ^ (line: 3)

This doesn't help more than:

mapping values are not allowed here
  in "scanner_error.yaml", line 3, column 23 

Conclusion

Using it involves a lot of adaptations in the code and we:

• Don't gain better messages. They point to lines and columns, but not the right ones.
• Need to patch the library in order to make indentation checks optional.