Preflight filters parsed by Optionable class.

This makes the filters similar to output options.
- Adds coherence to error messages.
- Enable aliases (used the ones suggested by @leoheck)
Additionally now the README.md preflights documentation comes directly
from --help-preflights

CHANGELOG.md

@@ -8,6 +8,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 ### Added
 - pdf_pcb_print.output can be used instead of pdf_pcb_print.output_name
+- The filters now accept the following aliases (suggested by @leoheck):
+  - filter_msg -> filter
+  - error_number -> number
+  - regexp -> regex
 
 ## [0.5.0] - 2020-07-11
 ### Changed

README.md

@@ -47,15 +47,24 @@ This tells to Kiplot that this file is using version 1 of the format.
 
 ### The *preflight* section
 
-This section is used to specify tasks that will executed before generating any output. The available tasks are:
+This section is used to specify tasks that will be executed before generating any output.
+
+#### Supported preflight options:
+
+- check_zone_fills: [boolean=false] Zones are filled before doing any operation involving PCB layers.
+- filters: [list(dict)] A list of entries to filter out ERC/DRC messages.
+  * Valid keys:
+    - *error_number*: Alias for number.
+    - `filter`: [string=''] Name for the filter, for documentation purposes.
+    - *filter_msg*: Alias for filter.
+    - `number`: [number=0] Error number we want to exclude.
+    - `regex`: [string='None'] Regular expression to match the text for the error we want to exclude.
+    - *regexp*: Alias for regex.
+- ignore_unconnected: [boolean=false] Option for `run_drc`. Ignores the unconnected nets. Useful if you didn't finish the routing.
+- run_drc: [boolean=false] Runs the DRC (Distance Rules Check). To ensure we have a valid PCB.
+- run_erc: [boolean=false] Runs the ERC (Electrical Rules Check). To ensure the schematic is electrically correct.
+- update_xml: [boolean=false] Update the XML version of the BoM (Bill of Materials). To ensure our generated BoM is up to date.
 
-- `run_erc` To run the ERC (Electrical Rules Check). To ensure the schematic is electrically correct. 
-- `run_drc` To run the DRC (Distance Rules Check). To ensure we have a valid PCB.
-- `update_xml` To update the XML version of the BoM (Bill of Materials). To ensure our generated BoM is up to date.
-- `check_zone_fills` Zones are filled before doing any operation involving PCB layers.
- 
-The `run_drc` command has the following option:
-- `ignore_unconnected` Ignores the unconnected nets. Useful if you didn't finish the routing.
 
 Here is an example of a *preflight* section:
 

docs/README.in

@@ -47,15 +47,9 @@ This tells to Kiplot that this file is using version 1 of the format.
 
 ### The *preflight* section
 
-This section is used to specify tasks that will executed before generating any output. The available tasks are:
+This section is used to specify tasks that will be executed before generating any output.
 
-- `run_erc` To run the ERC (Electrical Rules Check). To ensure the schematic is electrically correct. 
+#### @preflight@
-- `run_drc` To run the DRC (Distance Rules Check). To ensure we have a valid PCB.
-- `update_xml` To update the XML version of the BoM (Bill of Materials). To ensure our generated BoM is up to date.
-- `check_zone_fills` Zones are filled before doing any operation involving PCB layers.
- 
-The `run_drc` command has the following option:
-- `ignore_unconnected` Ignores the unconnected nets. Useful if you didn't finish the routing.
 
 Here is an example of a *preflight* section:
 

docs/replace_tags.pl

@@ -2,11 +2,13 @@
 
 $outputs =`../src/kiplot --help-outputs`;
 $cmd_help=`../src/kiplot --help`;
+$preflight=`../src/kiplot --help-preflights`;
 
 while (<>)
   {
    $_ =~ s/\@outputs\@/$outputs/;
    $_ =~ s/\@cmd_help\@/$cmd_help/;
+   $_ =~ s/\@preflight\@/$preflight/;
    print $_;
   }
 

docs/samples/generic_plot.kiplot.yaml

@@ -4,7 +4,7 @@ kiplot:
 preflight:
   # [boolean=false] Zones are filled before doing any operation involving PCB layers
   check_zone_fills: true
-  # A list of entries to filter out ERC/DRC messages. Keys: `filter`, `number` and `regex`
+  # [list(dict)] A list of entries to filter out ERC/DRC messages
   filters: 
     - filter: 'Filter description'
       number: 10

kiplot/config_reader.py

@@ -68,35 +68,6 @@ class CfgYamlReader(object):
 
         return o_out
 
-    def _parse_filters(self, filters):
-        if not isinstance(filters, list):
-            config_error("'filters' must be a list")
-        parsed = None
-        for filter in filters:
-            if 'filter' in filter:
-                comment = filter['filter']
-                if 'number' in filter:
-                    number = filter['number']
-                    if number is None:
-                        config_error("empty 'number' in 'filter' definition ("+str(filter)+")")
-                else:
-                    config_error("missing 'number' for 'filter' definition ("+str(filter)+")")
-                if 'regex' in filter:
-                    regex = filter['regex']
-                    if regex is None:
-                        config_error("empty 'regex' in 'filter' definition ("+str(filter)+")")
-                else:
-                    config_error("missing 'regex' for 'filter' definition ("+str(filter)+")")
-                logger.debug("Adding DRC/ERC filter '{}','{}','{}'".format(comment, number, regex))
-                if parsed is None:
-                    parsed = ''
-                if comment:
-                    parsed += '# '+comment+'\n'
-                parsed += '{},{}\n'.format(number, regex)
-            else:
-                config_error("'filters' section of 'preflight' must contain 'filter' definitions (not "+str(filter)+")")
-        return parsed
-
     def _parse_preflight(self, pf):
         logger.debug("Parsing preflight options: {}".format(pf))
         if not isinstance(pf, dict):
@@ -107,8 +78,6 @@ class CfgYamlReader(object):
                 config_error("Unknown preflight: `{}`".format(k))
             try:
                 logger.debug("Parsing preflight "+k)
-                if k == 'filters':
-                    v = self._parse_filters(v)
                 o_pre = BasePreFlight.get_class_for(k)(k, v)
             except KiPlotConfigurationError as e:
                 config_error("In preflight '"+k+"': "+str(e))
@@ -241,11 +210,13 @@ def print_preflights_help():
     pres = BasePreFlight.get_registered()
     logger.debug('{} supported preflights'.format(len(pres)))
     print('Supported preflight options:\n')
-    for n, o in pres.items():
+    for n, o in OrderedDict(sorted(pres.items())).items():
-        help = o.__doc__
+        help, options = o.get_doc()
         if help is None:
             help = 'Undocumented'  # pragma: no cover
-        print('- {}: {}.'.format(n, help.rstrip()))
+        print('- {}: {}.'.format(n, help.strip()))
+        if options:
+            print_output_options(n, options, 2)
 
 
 def print_example_options(f, cls, name, indent, po):

kiplot/pre_base.py

@@ -88,6 +88,10 @@ class BasePreFlight(object):
         """ Returns a YAML value for the example config """
         return 'true'
 
+    @classmethod
+    def get_doc(cls):
+        return cls.__doc__, None
+
     def run(self):
         pass
 

kiplot/pre_filters.py

@@ -1,18 +1,78 @@
 import os
-from .gs import (GS)
+from kiplot.gs import GS
-from kiplot.macros import macros, pre_class  # noqa: F401
+from kiplot.error import KiPlotConfigurationError
+from kiplot.optionable import Optionable
+from kiplot.macros import macros, document, pre_class  # noqa: F401
+from kiplot.log import get_logger
+
+logger = get_logger(__name__)
+
+
+class FilterOptions(Optionable):
+    """ Valid options for a filter entry """
+    def __init__(self):
+        super().__init__()
+        self._unkown_is_error = True
+        with document:
+            self.filter = ''
+            """ Name for the filter, for documentation purposes """
+            self.filter_msg = None
+            """ {filter} """
+            self.number = 0
+            """ Error number we want to exclude """
+            self.error_number = None
+            """ {number} """
+            self.regex = 'None'
+            """ Regular expression to match the text for the error we want to exclude """
+            self.regexp = None
+            """ {regex} """  # pragma: no cover
+
+
+class FiltersOptions(Optionable):
+    """ A list of filter entries """
+    def __init__(self):
+        super().__init__()
+        with document:
+            self.filters = FilterOptions
+            """ [list(dict)] DRC/ERC errors to be ignored """  # pragma: no cover
+
+    def config(self, tree):
+        super().config(tree)
+        parsed = None
+        for f in self.filters:
+            where = ' (in `{}` filter)'.format(f.filter) if f.filter else ''
+            number = f.number
+            if not number:
+                raise KiPlotConfigurationError('Missing `number`'+where)
+            regex = f.regex
+            if regex == 'None':
+                raise KiPlotConfigurationError('Missing `regex`'+where)
+            comment = f.filter
+            logger.debug("Adding DRC/ERC filter '{}','{}','{}'".format(comment, number, regex))
+            if parsed is None:
+                parsed = ''
+            if comment:
+                parsed += '# '+comment+'\n'
+            parsed += '{},{}\n'.format(number, regex)
+        self.filters = parsed
 
 
 @pre_class
 class Filters(BasePreFlight):  # noqa: F821
-    """ A list of entries to filter out ERC/DRC messages. Keys: `filter`, `number` and `regex` """
+    """ [list(dict)] A list of entries to filter out ERC/DRC messages """
     def __init__(self, name, value):
-        super().__init__(name, value)
+        f = FiltersOptions()
+        f.config({'filters': value})
+        super().__init__(name, f.filters)
 
     def get_example():
         """ Returns a YAML value for the example config """
         return "\n    - filter: 'Filter description'\n      number: 10\n      regex: 'Regular expression to match'"
 
+    @classmethod
+    def get_doc(cls):
+        return cls.__doc__, FilterOptions
+
     def apply(self):
         # Create the filters file
         if self._value:

tests/test_plot/test_yaml_errors.py

@@ -276,42 +276,42 @@ def test_error_step_min_distance():
 def test_filter_not_list():
     ctx = context.TestContext('FilterNotList', PRJ, 'error_filter_not_list', '')
     ctx.run(EXIT_BAD_CONFIG)
-    assert ctx.search_err(".?filters.? must be a list ")
+    assert ctx.search_err("Option .?filters.? must be any of")
     ctx.clean_up()
 
 
 def test_filter_no_number():
     ctx = context.TestContext('FilterNoNumber', PRJ, 'error_filter_no_number', '')
     ctx.run(EXIT_BAD_CONFIG)
-    assert ctx.search_err("empty .?number.? in .?filter.?")
+    assert ctx.search_err("Option .?number.? must be a number")
     ctx.clean_up()
 
 
 def test_filter_no_number_2():
     ctx = context.TestContext('FilterNoNumber2', PRJ, 'error_filter_no_number_2', '')
     ctx.run(EXIT_BAD_CONFIG)
-    assert ctx.search_err("missing .?number.? for .?filter.?")
+    assert ctx.search_err("Missing .?number.?")
     ctx.clean_up()
 
 
 def test_filter_no_regex():
     ctx = context.TestContext('FilterNoRegex', PRJ, 'error_filter_no_regex', '')
     ctx.run(EXIT_BAD_CONFIG)
-    assert ctx.search_err("empty .?regex.? in .?filter.?")
+    assert ctx.search_err("Option .?regex.? must be a string")
     ctx.clean_up()
 
 
 def test_filter_no_regex_2():
     ctx = context.TestContext('FilterNoRegex2', PRJ, 'error_filter_no_regex_2', '')
     ctx.run(EXIT_BAD_CONFIG)
-    assert ctx.search_err("missing .?regex.? for .?filter.?")
+    assert ctx.search_err("Missing .?regex.?")
     ctx.clean_up()
 
 
 def test_filter_wrong_entry():
     ctx = context.TestContext('FilterWrongEntry', PRJ, 'error_filter_wrong_entry', '')
     ctx.run(EXIT_BAD_CONFIG)
-    assert ctx.search_err(".?filters.? section of .?preflight.? must contain .?filter.?(.*)Pad 2 of C4")
+    assert ctx.search_err("Unknown option .?numerito.?")
     ctx.clean_up()
 
 

tests/yaml_samples/drc_filter.kiplot.yaml

@@ -8,8 +8,8 @@ preflight:
     - filter: 'Ignore C3 pad 2 too close to anything'
       number: 4
       regex:  'Pad 2 of C3'
-    - filter: 'Ignore unconnected pad 2 of C4'
+    - filter_msg: 'Ignore unconnected pad 2 of C4'
-      number: 2
+      error_number: 2
-      regex:  'Pad 2 of C4'
+      regexp:  'Pad 2 of C4'
 
 

tests/yaml_samples/error_filter_wrong_entry.kiplot.yaml

@@ -9,7 +9,7 @@ preflight:
       filter: 'Ignore C3 pad 2 too close to anything'
       regex:  'Pad 2 of C3'
     - regex:  'Pad 2 of C4'
-      number: 2
+      numerito: 2