[Blender Export] Added PCB3D generation

- Also control over which components are included
- Also highlight components

README.md

@@ -1556,11 +1556,13 @@ Notes:
 * Blender Export **Experimental**
   * Type: `blender_export`
   * Description: Exports the PCB in various 3D file formats.
-                 Also renders the PCB in high-quality.
+                 Also renders the PCB with high-quality.
                  This output is complex to setup and needs very big dependencies.
                  Please be patient when using it.
                  You need Blender with the pcb2blender plug-in installed.
-                 Visit: [pcb2blender](https://github.com/30350n/pcb2blender)
+                 Visit: [pcb2blender](https://github.com/30350n/pcb2blender).
+                 You can just generate the exported PCB if no output is specified.
+                 You can also export the PCB and render it at the same time
   * Valid keys:
     - **`comment`**: [string=''] A comment for documentation purposes. It helps to identify the output.
     - **`dir`**: [string='./'] Output directory for the generated files.
@@ -1568,11 +1570,26 @@ Notes:
     - **`name`**: [string=''] Used to identify this particular output definition.
                   Avoid using `_` as first character. These names are reserved for KiBot.
     - **`options`**: [dict] Options for the `blender_export` output.
+      * Valid keys:
+        - **`pcb3d`**: [string|dict] Options to export the PCB to Blender.
+                       You can also specify the name of the output that generates the PCB3D file.
+                       See the `PCB2Blender_2_1` and  `PCB2Blender_2_1_haschtl` templates.
           * Valid keys:
             - **`download`**: [boolean=true] Downloads missing 3D models from KiCad git. Only applies to models in KISYS3DMOD.
             - **`no_virtual`**: [boolean=false] Used to exclude 3D models for components with 'virtual' attribute.
-        - **`pcb3d`**: [string=''] Name of the output that generated the PCB3D file to import in Blender.
+            - **`show_components`**: [list(string)|string=all] [none,all] List of components to draw, can be also a string for `none` or `all`.
-                       See the `PCB2Blender_2_1` and  `PCB2Blender_2_1_haschtl` templates.
+                                     Unlike the `pcbdraw` output, the default is `all`.
+            - `dnf_filter`: [string|list(string)='_none'] Name of the filter to mark components as not fitted.
+                            A short-cut to use for simple cases where a variant is an overkill.
+            - `highlight`: [list(string)=[]] List of components to highlight.
+            - `highlight_on_top`: [boolean=false] Highlight over the component (not under).
+            - `highlight_padding`: [number=1.5] [0,1000] How much the highlight extends around the component [mm].
+            - `kicad_3d_url`: [string='https://gitlab.com/kicad/libraries/kicad-packages3D/-/raw/master/'] Base URL for the KiCad 3D models.
+            - `output`: [string='%f-%i%I%v.%x'] Name for the generated PCB3D file (%i='blender_export' %x='pcb3d'). Affected by global options.
+            - `pre_transform`: [string|list(string)='_none'] Name of the filter to transform fields before applying other filters.
+                               A short-cut to use for simple cases where a variant is an overkill.
+            - `variant`: [string=''] Board variant to apply.
+            - `version`: [string='2.1'] [2.1,2.1_haschtl] Variant of the format used.
         - **`render_options`**: [dict] How the render is done for the `render` output type.
           * Valid keys:
             - **`samples`**: [number=10] How many samples we create. Each sample is a raytracing render.
@@ -1593,9 +1610,6 @@ Notes:
             - `pos_x`: [number|string] X position [m]. You can use `width`, `height` and `size` for PCB dimensions.
             - `pos_y`: [number|string] Y position [m]. You can use `width`, `height` and `size` for PCB dimensions.
             - `pos_z`: [number|string] Z position [m]. You can use `width`, `height` and `size` for PCB dimensions.
-        - `dnf_filter`: [string|list(string)='_none'] Name of the filter to mark components as not fitted.
-                        A short-cut to use for simple cases where a variant is an overkill.
-        - `kicad_3d_url`: [string='https://gitlab.com/kicad/libraries/kicad-packages3D/-/raw/master/'] Base URL for the KiCad 3D models.
         - `light`: [dict|list(dict)] Options for the light/s.
           * Valid keys:
             - `name`: [string=''] Name for the light.
@@ -1625,12 +1639,9 @@ Notes:
                                This option controls if we add it for none, all or only for THT/SMD pads with solder paste.
             - `stack_boards`: [boolean=true] Move the sub-PCBs to their relative position.
             - `texture_dpi`: [number=1016.0] [508-2032] Texture density in dots per inch.
-        - `pre_transform`: [string|list(string)='_none'] Name of the filter to transform fields before applying other filters.
-                           A short-cut to use for simple cases where a variant is an overkill.
         - `rotate_x`: [number=0] Angle to rotate the board in the X axis, positive is clockwise [degrees].
         - `rotate_y`: [number=0] Angle to rotate the board in the Y axis, positive is clockwise [degrees].
         - `rotate_z`: [number=0] Angle to rotate the board in the Z axis, positive is clockwise [degrees].
-        - `variant`: [string=''] Board variant to apply.
     - `category`: [string|list(string)=''] The category for this output. If not specified an internally defined category is used.
                   Categories looks like file system paths, i.e. **PCB/fabrication/gerber**.
                   The categories are currently used for `navigate_results`.
@@ -1929,7 +1940,7 @@ Notes:
             - `dest`: [string=''] Destination directory inside the archive, empty means the same of the file.
             - `filter`: [string='.*'] A regular expression that source files must match.
             - `from_cwd`: [boolean=false] Use the current working directory instead of the dir specified by `-d`.
-            - `from_output_dir`: [boolean=false] Use the current the directory specified by the output instead of the dir specified by `-d`.
+            - `from_output_dir`: [boolean=false] Use the current directory specified by the output instead of the dir specified by `-d`.
                                  Note that it only applies when using `from_output` and no `dest` is specified.
                                  It has more prescedence than `from_cwd`.
         - **`format`**: [string='ZIP'] [ZIP,TAR,RAR] Output file format.

docs/samples/generic_plot.kibot.yaml

@@ -93,11 +93,13 @@ preflight:
 
 outputs:
   # Blender Export **Experimental**:
-  # Also renders the PCB in high-quality.
+  # Also renders the PCB with high-quality.
   # This output is complex to setup and needs very big dependencies.
   # Please be patient when using it.
   # You need Blender with the pcb2blender plug-in installed.
-  # Visit: [pcb2blender](https://github.com/30350n/pcb2blender)
+  # Visit: [pcb2blender](https://github.com/30350n/pcb2blender).
+  # You can just generate the exported PCB if no output is specified.
+  # You can also export the PCB and render it at the same time
   - name: 'blender_export_example'
     comment: 'Exports the PCB in various 3D file formats.'
     type: 'blender_export'
@@ -117,13 +119,6 @@ outputs:
         pos_y: 0
         # [number|string] Z position [m]. You can use `width`, `height` and `size` for PCB dimensions
         pos_z: 0
-      # [string|list(string)='_none'] Name of the filter to mark components as not fitted.
-      # A short-cut to use for simple cases where a variant is an overkill
-      dnf_filter: '_none'
-      # [boolean=true] Downloads missing 3D models from KiCad git. Only applies to models in KISYS3DMOD
-      download: true
-      # [string='https://gitlab.com/kicad/libraries/kicad-packages3D/-/raw/master/'] Base URL for the KiCad 3D models
-      kicad_3d_url: 'https://gitlab.com/kicad/libraries/kicad-packages3D/-/raw/master/'
       # [dict|list(dict)] Options for the light/s
       light:
         # [string=''] Name for the light
@@ -134,8 +129,6 @@ outputs:
           pos_y: 0
           # [number|string] Z position [m]. You can use `width`, `height` and `size` for PCB dimensions
           pos_z: 0
-      # [boolean=false] Used to exclude 3D models for components with 'virtual' attribute
-      no_virtual: false
       # [dict|list(dict)] Outputs to generate in the same run
       outputs:
         # [string='%f-%i%I%v.%x'] Name for the generated file (%i='3D_blender_$VIEW' %x=VARIABLE).
@@ -149,9 +142,37 @@ outputs:
           # Note that some formats includes the light and camera and others are just the 3D model
           # (i.e. STL and PLY)
           type: 'render'
-      # [string=''] Name of the output that generated the PCB3D file to import in Blender.
+      # [string|dict] Options to export the PCB to Blender.
+      # You can also specify the name of the output that generates the PCB3D file.
       # See the `PCB2Blender_2_1` and  `PCB2Blender_2_1_haschtl` templates
-      pcb3d: ''
+      pcb3d:
+        # [string|list(string)='_none'] Name of the filter to mark components as not fitted.
+        # A short-cut to use for simple cases where a variant is an overkill
+        dnf_filter: '_none'
+        # [boolean=true] Downloads missing 3D models from KiCad git. Only applies to models in KISYS3DMOD
+        download: true
+        # [list(string)=[]] List of components to highlight
+        highlight: []
+        # [boolean=false] Highlight over the component (not under)
+        highlight_on_top: false
+        # [number=1.5] [0,1000] How much the highlight extends around the component [mm]
+        highlight_padding: 1.5
+        # [string='https://gitlab.com/kicad/libraries/kicad-packages3D/-/raw/master/'] Base URL for the KiCad 3D models
+        kicad_3d_url: 'https://gitlab.com/kicad/libraries/kicad-packages3D/-/raw/master/'
+        # [boolean=false] Used to exclude 3D models for components with 'virtual' attribute
+        no_virtual: false
+        # [string='%f-%i%I%v.%x'] Name for the generated PCB3D file (%i='blender_export' %x='pcb3d'). Affected by global options
+        output: '%f-%i%I%v.%x'
+        # [string|list(string)='_none'] Name of the filter to transform fields before applying other filters.
+        # A short-cut to use for simple cases where a variant is an overkill
+        pre_transform: '_none'
+        # [list(string)|string=all] [none,all] List of components to draw, can be also a string for `none` or `all`.
+        # Unlike the `pcbdraw` output, the default is `all`
+        show_components: all
+        # [string=''] Board variant to apply
+        variant: ''
+        # [string='2.1'] [2.1,2.1_haschtl] Variant of the format used
+        version: '2.1'
       # Options to configure how Blender imports the PCB.
       # The default values are good for most cases
       pcb_import:
@@ -172,9 +193,6 @@ outputs:
         stack_boards: true
         # [number=1016.0] [508-2032] Texture density in dots per inch
         texture_dpi: 1016.0
-      # [string|list(string)='_none'] Name of the filter to transform fields before applying other filters.
-      # A short-cut to use for simple cases where a variant is an overkill
-      pre_transform: '_none'
       # [dict] How the render is done for the `render` output type
       render_options:
         # [string='#66667F'] First color for the background gradient
@@ -196,8 +214,6 @@ outputs:
       rotate_y: 0
       # [number=0] Angle to rotate the board in the Z axis, positive is clockwise [degrees]
       rotate_z: 0
-      # [string=''] Board variant to apply
-      variant: ''
       # [string='top'] [top,bottom,front,rear,right,left,z,Z,y,Y,x,X] Point of view.
       # Compatible with `render_3d`
       view: 'top'
@@ -561,7 +577,7 @@ outputs:
           # [string=''] Collect files from the selected output.
           # When used the `source` option is ignored
           from_output: ''
-          # [boolean=false] Use the current the directory specified by the output instead of the dir specified by `-d`.
+          # [boolean=false] Use the current directory specified by the output instead of the dir specified by `-d`.
           # Note that it only applies when using `from_output` and no `dest` is specified.
           # It has more prescedence than `from_cwd`
           from_output_dir: false

kibot/out_base.py

@@ -242,6 +242,11 @@ class VariantOptions(BaseOptions):
         self.dnf_filter = BaseFilter.solve_filter(self.dnf_filter, 'dnf_filter')
         self.pre_transform = BaseFilter.solve_filter(self.pre_transform, 'pre_transform', is_transform=True)
 
+    def copy_options(self, ref):
+        self.variant = ref.variant
+        self.dnf_filter = ref.dnf_filter
+        self.pre_transform = ref.pre_transform
+
     def get_refs_hash(self):
         if not self._comps:
             return None

kibot/out_base_3d.py

@@ -71,6 +71,12 @@ class Base3DOptions(VariantOptions):
         super().__init__()
         self._expand_id = '3D'
 
+    def copy_options(self, ref):
+        super().copy_options(ref)
+        self.no_virtual = ref.no_virtual
+        self.download = ref.download
+        self.kicad_3d_url = ref.kicad_3d_url
+
     def download_model(self, url, fname, rel_dirs):
         """ Download the 3D model from the provided URL """
         logger.debug('Downloading `{}`'.format(url))
@@ -268,6 +274,16 @@ class Base3DOptionsWithHL(Base3DOptions):
         else:
             self.highlight = self.solve_kf_filters(self.highlight)
 
+    def copy_options(self, ref):
+        """ Copy its options from another similar object """
+        super().copy_options(ref)
+        self.show_components = ref.show_components
+        self.highlight = ref.highlight
+        self.highlight_padding = ref.highlight_padding
+        self.highlight_on_top = ref.highlight_on_top
+        self._filters_to_expand = ref._filters_to_expand
+        self._show_all_components = ref._show_all_components
+
     def apply_show_components(self):
         if self._show_all_components:
             # Don't change anything

kibot/out_blender_export.py

@@ -12,12 +12,13 @@ Dependencies:
 from copy import copy
 import json
 import os
-from tempfile import NamedTemporaryFile
+from tempfile import NamedTemporaryFile, TemporaryDirectory
 from .error import KiPlotConfigurationError
-from .kiplot import get_output_targets, run_output, run_command, register_xmp_import
+from .kiplot import get_output_targets, run_output, run_command, register_xmp_import, config_output
 from .gs import GS
 from .optionable import Optionable
-from .out_base_3d import Base3DOptions, Base3D
+from .out_base_3d import Base3D, Base3DOptionsWithHL
+from .registrable import RegOutput
 from .macros import macros, document, output_class  # noqa: F401
 from . import log
 
@@ -133,14 +134,29 @@ class BlenderRenderOptions(Optionable):
         self._unkown_is_error = True
 
 
-class Blender_ExportOptions(Base3DOptions):
+class PCB3DExportOptions(Base3DOptionsWithHL):
+    """ Options to generate the PCB3D file """
+    def __init__(self, field=None):
+        super().__init__()
+        with document:
+            self.output = GS.def_global_output
+            """ Name for the generated PCB3D file (%i='blender_export' %x='pcb3d') """
+            self.version = '2.1'
+            """ [2.1,2.1_haschtl] Variant of the format used """
+        self._expand_id = 'blender_export'
+        self._expand_ext = 'pcb3d'
+        self._unkown_is_error = True
+
+
+class Blender_ExportOptions(Optionable):
     _views = {'top': 'z', 'bottom': 'Z', 'front': 'y', 'rear': 'Y', 'right': 'x', 'left': 'X'}
     _rviews = {v: k for k, v in _views.items()}
 
     def __init__(self):
         with document:
-            self.pcb3d = ""
+            self.pcb3d = PCB3DExportOptions
-            """ *Name of the output that generated the PCB3D file to import in Blender.
+            """ *[string|dict] Options to export the PCB to Blender.
+                You can also specify the name of the output that generates the PCB3D file.
                 See the `PCB2Blender_2_1` and  `PCB2Blender_2_1_haschtl` templates """
             self.pcb_import = PCB2BlenderOptions
             """ Options to configure how Blender imports the PCB.
@@ -173,11 +189,12 @@ class Blender_ExportOptions(Base3DOptions):
     def config(self, parent):
         super().config(parent)
         # Check we at least have a name for the source output
-        if not self.pcb3d:
+        if isinstance(self.pcb3d, type) or (isinstance(self.pcb3d, str) and not self.pcb3d):
-            raise KiPlotConfigurationError('You must specify the name of the output that generates the PCB3D file')
+            raise KiPlotConfigurationError('You must specify the name of the output that'
+                                           ' generates the PCB3D file or its options')
         # Do we have outputs?
         if isinstance(self.outputs, type):
-            raise KiPlotConfigurationError('You must specify at least one output')
+            self.outputs = []
         elif isinstance(self.outputs, BlenderOutputOptions):
             # One, make a list
             self.outputs = [self.outputs]
@@ -238,17 +255,135 @@ class Blender_ExportOptions(Base3DOptions):
     def get_targets(self, out_dir):
         return [self.get_output_filename(o, out_dir) for o in self.outputs]
 
-    def run(self, output):
+    def create_vrml(self, dest_dir):
-        super().run(output)
+        tree = {'name': '_temporal_vrml_for_pcb3d',
-        command = self.ensure_tool('Blender')
+                'type': 'vrml',
-        pcb3d_targets, pcb3d_out_dir, pcb3d_out = get_output_targets(self.pcb3d, self._parent)
+                'comment': 'Internally created for the PCB3D',
+                'dir': dest_dir,
+                'options': {'output': 'pcb.wrl',
+                            'dir_models': 'components',
+                            'use_pcb_center_as_ref': False,
+                            'model_units': 'meters'}}
+        out = RegOutput.get_class_for('vrml')()
+        out.set_tree(tree)
+        config_output(out)
+        out.options.copy_options(self.pcb3d)
+        logger.debug(' - Creating VRML ...')
+        out.options.run(os.path.join(dest_dir, 'pcb.wrl'))
+
+    def create_layers(self, dest_dir):
+        out_dir = os.path.join(dest_dir, 'layers')
+        tree = {'name': '_temporal_svgs_layers',
+                'type': 'svg',
+                'comment': 'Internally created for the PCB3D',
+                'dir': out_dir,
+                'options': {'output': '%i.%x',
+                            'margin': 1,
+                            'limit_viewbox': True,
+                            'svg_precision': 6,
+                            'drill_marks': 'none'},
+                'layers': ['F.Cu', 'B.Cu', 'F.Paste', 'B.Paste', 'F.Mask', 'B.Mask',
+                           {'layer': 'F.SilkS', 'suffix': 'F_SilkS'},
+                           {'layer': 'B.SilkS', 'suffix': 'B_SilkS'}]}
+        out = RegOutput.get_class_for(tree['type'])()
+        out.set_tree(tree)
+        config_output(out)
+        logger.debug(' - Creating SVG for layers ...')
+        out.run(out_dir)
+
+    def create_pads(self, dest_dir):
+        tree = {'name': '_temporal_pcb3d_tools',
+                'type': 'pcb2blender_tools',
+                'comment': 'Internally created for the PCB3D',
+                'dir': dest_dir,
+                'options': {'stackup_create': self.pcb3d.version == '2.1_haschtl'}}
+        out = RegOutput.get_class_for(tree['type'])()
+        out.set_tree(tree)
+        config_output(out)
+        logger.debug(' - Creating Pads and boundary ...')
+        out.run(dest_dir)
+
+    def create_pcb3d(self, data_dir):
+        out_dir = self._parent.output_dir
+        # Compute the name for the PCB3D
+        cur_id = self._expand_id
+        cur_ext = self._expand_ext
+        self._expand_id = self.pcb3d._expand_id
+        self._expand_ext = self.pcb3d._expand_ext
+        out_name = self._parent.expand_filename(out_dir, self.pcb3d.output)
+        self._expand_id = cur_id
+        self._expand_ext = cur_ext
+        tree = {'name': '_temporal_compress_pcb3d',
+                'type': 'compress',
+                'comment': 'Internally created for the PCB3D',
+                'dir': out_dir,
+                'options': {'output': out_name,
+                            'format': 'ZIP',
+                            'files': [{'source': os.path.join(data_dir, 'boards'),
+                                       'dest': '/'},
+                                      {'source': os.path.join(data_dir, 'boards/*'),
+                                       'dest': 'boards'},
+                                      {'source': os.path.join(data_dir, 'components'),
+                                       'dest': '/'},
+                                      {'source': os.path.join(data_dir, 'components/*'),
+                                       'dest': 'components'},
+                                      {'source': os.path.join(data_dir, 'layers'),
+                                       'dest': '/'},
+                                      {'source': os.path.join(data_dir, 'layers/*'),
+                                       'dest': 'layers'},
+                                      {'source': os.path.join(data_dir, 'pads'),
+                                       'dest': '/'},
+                                      {'source': os.path.join(data_dir, 'pads/*'),
+                                       'dest': 'pads'},
+                                      {'source': os.path.join(data_dir, 'pcb.wrl'),
+                                       'dest': '/'},
+                                      ]}}
+        out = RegOutput.get_class_for(tree['type'])()
+        out.set_tree(tree)
+        config_output(out)
+        logger.debug(' - Creating the PCB3D ...')
+        out.run(out_dir)
+        return out_name
+
+    def solve_pcb3d(self):
+        if isinstance(self.pcb3d, str):
+            # An output creates it
+            pcb3d_targets, _, pcb3d_out = get_output_targets(self.pcb3d, self._parent)
             pcb3d_file = pcb3d_targets[0]
             logger.debug('- From file '+pcb3d_file)
             if not pcb3d_out._done:
                 logger.debug('-  Running '+self.pcb3d)
                 run_output(pcb3d_out)
+            self._pcb3d = PCB3DExportOptions()
+            self._pcb3d.output = pcb3d_file
+            # Needed by ensure tool
+            self._pcb3d._parent = self._parent
+        else:
+            # We create it
+            with TemporaryDirectory() as tmp_dir:
+                # VRML
+                self.create_vrml(tmp_dir)
+                # SVG layers
+                self.create_layers(tmp_dir)
+                # Pads and bounds
+                self.create_pads(tmp_dir)
+                # Compress the files
+                pcb3d_file = self.create_pcb3d(tmp_dir)
+            self._pcb3d = self.pcb3d
+            # Needed by ensure tool
+            self.type = self._parent.type
         if not os.path.isfile(pcb3d_file):
             raise KiPlotConfigurationError('Missing '+pcb3d_file)
+        return pcb3d_file
+
+    def run(self, output):
+        pcb3d_file = self.solve_pcb3d()
+        # If no outputs specified just finish
+        # Can be used to export the PCB to Blender
+        if not self.outputs:
+            return
+        # Make sure Blender is available
+        command = self._pcb3d.ensure_tool('Blender')
         # Create a JSON with the scene information
         with NamedTemporaryFile(mode='w', suffix='.json') as f:
             scene = {}
@@ -312,11 +447,13 @@ class Blender_ExportOptions(Base3DOptions):
 class Blender_Export(Base3D):
     """ Blender Export **Experimental**
         Exports the PCB in various 3D file formats.
-        Also renders the PCB in high-quality.
+        Also renders the PCB with high-quality.
         This output is complex to setup and needs very big dependencies.
         Please be patient when using it.
         You need Blender with the pcb2blender plug-in installed.
-        Visit: [pcb2blender](https://github.com/30350n/pcb2blender) """
+        Visit: [pcb2blender](https://github.com/30350n/pcb2blender).
+        You can just generate the exported PCB if no output is specified.
+        You can also export the PCB and render it at the same time """
     def __init__(self):
         super().__init__()
         with document:

kibot/out_compress.py

@@ -45,7 +45,7 @@ class FilesList(Optionable):
             self.from_cwd = False
             """ Use the current working directory instead of the dir specified by `-d` """
             self.from_output_dir = False
-            """ Use the current the directory specified by the output instead of the dir specified by `-d`.
+            """ Use the current directory specified by the output instead of the dir specified by `-d`.
                 Note that it only applies when using `from_output` and no `dest` is specified.
                 It has more prescedence than `from_cwd` """
             self.from_output = ''

tests/yaml_samples/blender_export_2.kibot.yaml

@@ -0,0 +1,30 @@
+# KiBot Blender export test 2
+# Generating the PCB3D in the blender_export
+# src/kibot -vv -b tests/data/ArduinoLearningKitStarter.kicad_pcb -c tests/yaml_samples/blender_export_1.kibot.yaml -d r3d_lst
+kibot:
+  version: 1
+
+outputs:
+  - name: '3d_export'
+    comment: "Exports the PCB in blender format"
+    type: blender_export
+    options:
+      pcb3d:
+        show_components: ["RV1", "RV2", "U1", "U2", "U3"]
+        highlight: ["RV1"]
+      rotate_x: 30
+      rotate_z: -20
+      # view: bottom
+      render_options:
+        transparent_background: true
+        samples: 10
+        #resolution_x: 1920
+        #resolution_y: 1080
+      outputs:
+        - type: blender
+        - type: render
+
+  - name: 'navigate'
+    comment: "Browse the results"
+    type: navigate_results
+    run_by_default: false