Merged the kicost_dk branch:

- New internal BoM XLSX options to control KiCost:
  - Dis/Enable APIs
  - Add a description column
  - Specs worksheet generation
  - Which spec columns
  - Specify a KiCost config file
- Passed debug level to KiCost
- Added support for KiCost log style to our logger

README.md

@@ -944,9 +944,37 @@ Next time you need this list just use an alias, like this:
             - `hide_stats_info`: [boolean=false] Hide statistics information.
             - `highlight_empty`: [boolean=true] Use a color for empty cells. Applies only when `col_colors` is `true`.
             - `kicost`: [boolean=false] Enable KiCost worksheet creation.
+            - `kicost_api_disable`: [string|list(string)=''] List of KiCost APIs to disable.
+            - `kicost_api_enable`: [string|list(string)=''] List of KiCost APIs to enable.
+            - `kicost_config`: [string=''] KiCost configuration file. It contains the keys for the different distributors APIs.
+                               The regular KiCost config is used when empty.
+            - `kicost_dist_desc`: [boolean=false] Used to add a column with the distributor's description. So you can check this is the right component.
             - `logo`: [string|boolean=''] PNG file to use as logo, use false to remove.
             - `logo_scale`: [number=2] Scaling factor for the logo. Note that this value isn't honored by all spreadsheet software.
             - `max_col_width`: [number=60] [20,999] Maximum column width (characters).
+            - `specs`: [boolean=false] Enable Specs worksheet creation. Contains specifications for the components.
+                       Works with only some KiCost APIs.
+            - `specs_columns`: [list(dict)|list(string)] Which columns are included in the Specs worksheet. Use `References` for the references,
+                               'Row' for the order and 'Sep' to separate groups at the same level. By default all are included.
+                               Column names are distributor specific, the following aren't: '_desc', '_value', '_tolerance', '_footprint',
+                               '_power', '_current', '_voltage', '_frequency', '_temp_coeff', '_manf', '_size'.
+              * Valid keys:
+                - `comment`: [string=''] Used as explanation for this column. The XLSX output uses it.
+                - `field`: [string=''] Name of the field to use for this column.
+                - `join`: [list(dict)|list(string)|string=''] List of fields to join to this column.
+                  * Valid keys:
+                    - `field`: [string=''] Name of the field.
+                    - `text`: [string=''] Text to use instead of a field. This option is incompatible with the `field` option.
+                              Any space to separate it should be added in the text.
+                              Use \n for newline and \t for tab.
+                    - `text_after`: [string=''] Text to add after the field content. Will be added only if the field isn't empty.
+                                    Any space to separate it should be added in the text.
+                                    Use \n for newline and \t for tab.
+                    - `text_before`: [string=''] Text to add before the field content. Will be added only if the field isn't empty.
+                                     Any space to separate it should be added in the text.
+                                     Use \n for newline and \t for tab.
+                - `level`: [number=0] Used to group columns. The XLSX output uses it to collapse columns.
+                - `name`: [string=''] Name to display in the header. The field is used when empty.
             - `style`: [string='modern-blue'] Head style: modern-blue, modern-green, modern-red and classic.
             - `title`: [string='KiBot Bill of Materials'] BoM title.
     - `output_id`: [string=''] Text to use for the %I expansion content. To differentiate variations of this output.
@@ -2011,6 +2039,7 @@ Next time you need this list just use an alias, like this:
         - `dnf_filter`: [string|list(string)='_none'] Name of the filter to mark components as not fitted.
                         A short-cut to use for simple cases where a variant is an overkill.
         - `drill_marks`: [string='full'] What to use to indicate the drill places, can be none, small or full (for real scale).
+        - `enable_ki5_page_fix`: [boolean=true] Enable workaround for KiCad 5 bug.
         - `enable_ki6_page_fix`: [boolean=true] Enable workaround for KiCad 6 bug #11033.
         - `force_edge_cuts`: [boolean=true] Only useful for KiCad 6 when printing in one page, you can disable the edge here.
                              KiCad 5 forces it by default, and you can't control it from config files.

docs/samples/generic_plot.kibot.yaml

@@ -309,12 +309,53 @@ outputs:
         highlight_empty: true
         # [boolean=false] Enable KiCost worksheet creation
         kicost: false
+        # [string|list(string)=''] List of KiCost APIs to disable
+        kicost_api_disable: ''
+        # [string|list(string)=''] List of KiCost APIs to enable
+        kicost_api_enable: ''
+        # [string=''] KiCost configuration file. It contains the keys for the different distributors APIs.
+        # The regular KiCost config is used when empty
+        kicost_config: ''
+        # [boolean=false] Used to add a column with the distributor's description. So you can check this is the right component
+        kicost_dist_desc: false
         # [string|boolean=''] PNG file to use as logo, use false to remove
         logo: ''
         # [number=2] Scaling factor for the logo. Note that this value isn't honored by all spreadsheet software
         logo_scale: 2
         # [number=60] [20,999] Maximum column width (characters)
         max_col_width: 60
+        # [boolean=false] Enable Specs worksheet creation. Contains specifications for the components.
+        # Works with only some KiCost APIs
+        specs: false
+        # [list(dict)|list(string)] Which columns are included in the Specs worksheet. Use `References` for the references,
+        # 'Row' for the order and 'Sep' to separate groups at the same level. By default all are included.
+        # Column names are distributor specific, the following aren't: '_desc', '_value', '_tolerance', '_footprint',
+        # '_power', '_current', '_voltage', '_frequency', '_temp_coeff', '_manf', '_size'
+        specs_columns:
+          # [string=''] Used as explanation for this column. The XLSX output uses it
+          - comment: ''
+            # [string=''] Name of the field to use for this column
+            field: 'Row'
+            # [list(dict)|list(string)|string=''] List of fields to join to this column
+            join:
+              # [string=''] Name of the field
+              - field: 'Voltage'
+                # [string=''] Text to use instead of a field. This option is incompatible with the `field` option.
+                # Any space to separate it should be added in the text.
+                # Use \n for newline and \t for tab
+                text: ''
+                # [string=''] Text to add after the field content. Will be added only if the field isn't empty.
+                # Any space to separate it should be added in the text.
+                # Use \n for newline and \t for tab
+                text_after: ''
+                # [string=''] Text to add before the field content. Will be added only if the field isn't empty.
+                # Any space to separate it should be added in the text.
+                # Use \n for newline and \t for tab
+                text_before: ''
+            # [number=0] Used to group columns. The XLSX output uses it to collapse columns
+            level: 0
+            # [string=''] Name to display in the header. The field is used when empty
+            name: 'Line'
         # [string='modern-blue'] Head style: modern-blue, modern-green, modern-red and classic
         style: 'modern-blue'
         # [string='KiBot Bill of Materials'] BoM title
@@ -1433,6 +1474,8 @@ outputs:
       dnf_filter: '_none'
       # [string='full'] What to use to indicate the drill places, can be none, small or full (for real scale)
       drill_marks: 'full'
+      # [boolean=true] Enable workaround for KiCad 5 bug
+      enable_ki5_page_fix: true
       # [boolean=true] Enable workaround for KiCad 6 bug #11033
       enable_ki6_page_fix: true
       # [boolean=true] Only useful for KiCad 6 when printing in one page, you can disable the edge here.

kibot/bom/xlsx_writer.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
-# Copyright (c) 2020-2021 Salvador E. Tropea
-# Copyright (c) 2020-2021 Instituto Nacional de Tecnología Industrial
+# Copyright (c) 2020-2022 Salvador E. Tropea
+# Copyright (c) 2020-2022 Instituto Nacional de Tecnología Industrial
 # Copyright (c) 2016-2020 Oliver Henry Walters (@SchrodingersGat)
 # License: MIT
 # Project: KiBot (formerly KiPlot)
@@ -12,12 +12,16 @@ import io
 import pprint
 import os.path as op
 import sys
+import logging
 from textwrap import wrap
 from base64 import b64decode
+from math import ceil
 from .columnlist import ColumnList
 from .kibot_logo import KIBOT_LOGO
 from .. import log
-from ..misc import W_NOKICOST, W_UNKDIST
+from ..misc import W_NOKICOST, W_UNKDIST, KICOST_ERROR, W_BADFIELD
+from ..error import trace_dump
+from ..gs import GS
 from ..__main__ import __version__
 try:
     from xlsxwriter import Workbook
@@ -37,20 +41,27 @@ try:
             sys.path.insert(0, rel_path)
     # Init the logger first
     logger = log.get_logger()
-    from kicost.global_vars import set_logger
+    from kicost.global_vars import set_logger, KiCostError
     set_logger(logger)
     from kicost import PartGroup
     from kicost.kicost import query_part_info
     from kicost.spreadsheet import create_worksheet, Spreadsheet
     from kicost.distributors import (init_distributor_dict, set_distributors_logger, get_distributors_list,
-                                     get_dist_name_from_label, set_distributors_progress)
+                                     get_dist_name_from_label, set_distributors_progress, is_valid_api,
+                                     configure_from_environment, configure_apis)
     from kicost.edas import set_edas_logger
+    from kicost.edas.tools import partgroup_qty
+    from kicost.config import load_config
     # Progress mechanism: use the one declared in __main__ (TQDM)
     from kicost.__main__ import ProgressConsole
     set_distributors_progress(ProgressConsole)
     KICOST_SUPPORT = True
 except ModuleNotFoundError:
     KICOST_SUPPORT = False
+except ImportError:
+    logger.error("Installed KiCost is older than the version we support.")
+    logger.error("Try installing the last release or the current GIT code.")
+    KICOST_SUPPORT = False
 
 BG_GEN = "#E6FFEE"  # "#C6DFCE"
 BG_KICAD = "#FFE6B3"  # "#DFC693"
@@ -66,10 +77,11 @@ GREY_L = "#f3f3f3"
 HEAD_COLOR_R = "#982020"
 HEAD_COLOR_G = "#009879"
 HEAD_COLOR_B = "#0e4e8e"
-DEFAULT_FMT = {'text_wrap': True, 'align': 'center_across', 'valign': 'vjustify'}
+DEFAULT_FMT = {'text_wrap': True, 'align': 'center_across', 'valign': 'vcenter'}
 KICOST_COLUMNS = {'refs': ColumnList.COL_REFERENCE,
                   'desc': ColumnList.COL_DESCRIPTION,
                   'qty': ColumnList.COL_GRP_BUILD_QUANTITY}
+SPECS_GENERATED = {ColumnList.COL_REFERENCE_L, ColumnList.COL_ROW_NUMBER_L, 'sep'}
 
 
 def bg_color(col):
@@ -246,6 +258,75 @@ def create_color_ref(workbook, col_colors, hl_empty, fmt_cols, do_kicost, kicost
             worksheet.write_string(row, 0, label, format)
 
 
+def get_spec(part, name):
+    if name[0] != '_':
+        return part.specs.get(name, ['', ''])
+    name = name[1:]
+    for v in part.dd.values():
+        val = v.extra_info.get(name, None)
+        if val:
+            return [name, val]
+    return ['', '']
+
+
+def create_meta(workbook, name, columns, parts, fmt_head, fmt_cols, max_w, rename, levels, comments, join):
+    worksheet = workbook.add_worksheet(name)
+    col_w = []
+    row_h = 1
+    for c, col in enumerate(columns):
+        name = rename.get(col.lower(), col) if rename else col
+        worksheet.write_string(0, c, name, fmt_head)
+        text_l = max(len(col), 6)
+        if text_l > max_w:
+            h = len(wrap(col, max_w))
+            row_h = max(row_h, h)
+            text_l = max_w
+        col_w.append(text_l)
+    if row_h > 1:
+        worksheet.set_row(0, 15.0*row_h)
+    for r, part in enumerate(parts):
+        # Add the references as another spec
+        part.specs[ColumnList.COL_REFERENCE_L] = (ColumnList.COL_REFERENCE, part.collapsed_refs)
+        # Also add the row
+        part.specs[ColumnList.COL_ROW_NUMBER_L] = (ColumnList.COL_ROW_NUMBER, str(r+1))
+        row_h = 1
+        for c, col in enumerate(columns):
+            col_l = col.lower()
+            if col_l == 'sep':
+                col_w[c] = 0
+                continue
+            v = get_spec(part, col_l)
+            text = v[1]
+            # Append text from other fields
+            if join:
+                for j in join:
+                    if j[0] == col_l:
+                        for c_join in j[1:]:
+                            v = part.specs.get(c_join, None)
+                            if v:
+                                text += ' ' + v[1]
+            text_l = len(text)
+            if not text_l:
+                continue
+            fmt_kind = 0 if col_l in SPECS_GENERATED else 2
+            worksheet.write_string(r+1, c, text, fmt_cols[fmt_kind][r % 2])
+            if text_l > col_w[c]:
+                if text_l > max_w:
+                    h = len(wrap(text, max_w))
+                    row_h = max(row_h, h)
+                    text_l = max_w
+                col_w[c] = text_l
+        if row_h > 1:
+            worksheet.set_row(r+1, 15.0*row_h)
+    for i, width in enumerate(col_w):
+        ops = {'level': levels[i] if levels else 0}
+        if not width:
+            ops['hidden'] = 1
+        if comments and comments[i]:
+            worksheet.write_comment(0, i, comments[i])
+        worksheet.set_column(i, i, width, None, ops)
+
+
 def adjust_widths(worksheet, column_widths, max_width, levels):
     c_levels = len(levels)
     for i, width in enumerate(column_widths):
@@ -375,7 +456,9 @@ def remove_unknown_distributors(distributors, available, silent):
         d = d.lower()
         if d not in available:
             # Is the label of the column?
-            d = get_dist_name_from_label(d)
+            new_d = get_dist_name_from_label(d)
+            if new_d is not None:
+                d = new_d
         if d not in available:
             if not silent:
                 logger.warning(W_UNKDIST+'Unknown distributor `{}`'.format(d))
@@ -412,7 +495,51 @@ def compute_qtys(cfg, g):
     return [str(g.get_count(sch.name)) for sch in cfg.aggregate]
 
 
-def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_subtitle, cfg):
+def create_meta_sheets(workbook, used_parts, fmt_head, fmt_cols, cfg, ss):
+    if cfg.xlsx.specs:
+        meta_names = ['Specs', 'Specs (DNF)']
+        for ws in range(2):
+            spec_cols = {}
+            spec_cols_l = set()
+            parts = used_parts[ws]
+            for part in parts:
+                for name_l, v in part.specs.items():
+                    spec_cols_l.add(name_l)
+                    spec = v[0]
+                    if spec in spec_cols:
+                        spec_cols[spec] += 1
+                    else:
+                        spec_cols[spec] = 1
+            if len(spec_cols):
+                columns = cfg.xlsx.s_columns
+                if columns is None:
+                    # Use all columns, sort them by relevance (most used) and alphabetically
+                    c = len(parts)
+                    columns = sorted(spec_cols, key=lambda k: (c - spec_cols[k], k))
+                    columns.insert(0, ColumnList.COL_REFERENCE)
+                else:
+                    # Inform about missing columns
+                    for c in columns:
+                        col = c.lower()
+                        if ((col[0] == '_' and col[1:] not in ss.extra_info_display) or
+                           (col[0] != '_' and col not in spec_cols_l and col not in SPECS_GENERATED)):
+                            logger.warning(W_BADFIELD+'Invalid Specs column name `{}` {}'.format(c, col[1:]))
+                create_meta(workbook, meta_names[ws], columns, parts, fmt_head, fmt_cols, cfg.xlsx.max_col_width,
+                            cfg.xlsx.s_rename, cfg.xlsx.s_levels, cfg.xlsx.s_comments, cfg.xlsx.s_join)
+
+
+def adapt_column_names(cfg):
+    for id, v in Spreadsheet.GLOBAL_COLUMNS.items():
+        if id in KICOST_COLUMNS:
+            # We use another name
+            new_id = KICOST_COLUMNS[id]
+            v['label'] = new_id
+            id = new_id.lower()
+        if id in cfg.column_rename:
+            v['label'] = cfg.column_rename[id]
+
+
+def _create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_subtitle, fmt_head, fmt_cols, cfg):
     if not KICOST_SUPPORT:
         logger.warning(W_NOKICOST, 'KiCost sheet requested but failed to load KiCost support')
         return
@@ -426,6 +553,21 @@ def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_s
     # Force KiCost to use our logger
     set_distributors_logger(logger)
     set_edas_logger(logger)
+    if GS.debug_enabled:
+        logger.setLevel(logging.DEBUG+1-GS.debug_level)
+    # Load KiCost config (includes APIs config)
+    api_options = load_config(cfg.xlsx.kicost_config)
+    # Environment with overwrite
+    configure_from_environment(api_options, True)
+    # Filter which APIs we want
+    for api in cfg.xlsx.kicost_api_disable:
+        if is_valid_api(api):
+            api_options[api]['enable'] = False
+    for api in cfg.xlsx.kicost_api_enable:
+        if is_valid_api(api):
+            api_options[api]['enable'] = True
+    # Configure the APIs
+    configure_apis(api_options)
     # Start with a clean list of available distributors
     init_distributor_dict()
     # Create the projects information structure
@@ -437,6 +579,9 @@ def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_s
     Spreadsheet.ADJUST_ROW_AND_COL_SIZE = True
     Spreadsheet.MAX_COL_WIDTH = cfg.xlsx.max_col_width
     Spreadsheet.PART_NSEQ_SEPRTR = cfg.ref_separator
+    Spreadsheet.SUPPRESS_DIST_DESC = not cfg.xlsx.kicost_dist_desc
+    # Keep our sorting
+    Spreadsheet.SORT_GROUPS = False
     # Make the version less intrusive
     Spreadsheet.WRK_FORMATS['about_msg']['font_size'] = 8
     # Don 't add project info, we add our own data
@@ -467,14 +612,8 @@ def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_s
     # Pass any extra column
     adapt_extra_cost_columns(cfg)
     # Adapt the column names
-    for id, v in Spreadsheet.GLOBAL_COLUMNS.items():
-        if id in KICOST_COLUMNS:
-            # We use another name
-            new_id = KICOST_COLUMNS[id]
-            v['label'] = new_id
-            id = new_id.lower()
-        if id in cfg.column_rename:
-            v['label'] = cfg.column_rename[id]
+    adapt_column_names(cfg)
+    used_parts = []
     for ws in range(2):
         # Second pass is DNF
         dnf = ws == 1
@@ -483,6 +622,7 @@ def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_s
             break
         # Create the parts structure from the groups
         parts = []
+        multi_prj = len(prj_info) > 1
         for g in groups:
             if (cfg.ignore_dnf and not g.is_fitted()) != dnf:
                 continue
@@ -490,6 +630,16 @@ def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_s
             part.refs = [c.ref for c in g.components]
             part.fields = g.fields
             part.fields['manf#_qty'] = compute_qtys(cfg, g)
+            # Solve the QTYs
+            part.qty_str, part.qty = partgroup_qty(part)
+            # Total for all boards
+            if multi_prj:
+                total = 0
+                for i_prj, p_info in enumerate(prj_info):
+                    total += part.qty[i_prj] * p_info['qty']
+            else:
+                total = part.qty * prj_info[0]['qty']
+            part.qty_total_spreadsheet = ceil(total)
             parts.append(part)
             # Process any "join" request
             apply_join_requests(cfg.join_ce, part.fields, g.fields)
@@ -523,6 +673,9 @@ def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_s
         if cfg.xlsx.title:
             wks.set_row(0, 32)
             wks.merge_range(0, col1, 0, ss.globals_width, cfg.xlsx.title, fmt_title)
+        used_parts.append(parts)
+    # Specs sheets
+    create_meta_sheets(workbook, used_parts, fmt_head, fmt_cols, cfg, ss)
     colors = {}
     colors['Best price'] = ss.wrk_formats['best_price']
     colors['No manufacturer or distributor code'] = ss.wrk_formats['not_manf_codes']
@@ -536,6 +689,15 @@ def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_s
     return colors
 
 
+def create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_subtitle, fmt_head, fmt_cols, cfg):
+    try:
+        return _create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_subtitle, fmt_head, fmt_cols, cfg)
+    except KiCostError as e:
+        trace_dump()
+        logger.error('KiCost error: `{}` ({})'.format(e.msg, e.id))
+        exit(KICOST_ERROR)
+
+
 def write_xlsx(filename, groups, col_fields, head_names, cfg):
     """
     Write BoM out to a XLSX file
@@ -660,7 +822,8 @@ def write_xlsx(filename, groups, col_fields, head_names, cfg):
     # Optionally add KiCost information
     kicost_colors = None
     if cfg.xlsx.kicost:
-        kicost_colors = create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_subtitle, cfg)
+        kicost_colors = create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_subtitle, fmt_head,
+                                            fmt_cols, cfg)
     # Add a sheet for the color references
     create_color_ref(workbook, cfg.xlsx.col_colors, hl_empty, fmt_cols, cfg.xlsx.kicost and KICOST_SUPPORT, kicost_colors)
 

kibot/log.py

@@ -92,6 +92,16 @@ class MyLogger(logging.Logger):
         else:
             super().warning(buf, **kwargs)
 
+    def log(self, level, msg, *args, **kwargs):
+        if level < self.getEffectiveLevel():
+            return
+        if isinstance(msg, tuple):
+            msg = ' '.join(map(str, msg))
+        if sys.version_info >= (3, 8):
+            super(self.__class__, self).debug(msg, stacklevel=2, *args, **kwargs)  # pragma: no cover (Py38)
+        else:
+            super(self.__class__, self).debug(msg, *args, **kwargs)
+
     def log_totals(self):
         if MyLogger.warn_cnt:
             filt_msg = ''
@@ -144,7 +154,7 @@ class CustomFormatter(logging.Formatter):
     """Logging Formatter to add colors"""
 
     def __init__(self, stream):
-        super().__init__()
+        super(logging.Formatter, self).__init__()
         if stream.isatty():
             white = Fore.WHITE
             yellow = Fore.YELLOW + Style.BRIGHT

kibot/misc.py

@@ -36,6 +36,7 @@ CORRUPTED_SCH = 22
 WRONG_INSTALL = 23
 RENDER_3D_ERR = 24
 FAILED_EXECUTE = 25
+KICOST_ERROR = 26
 error_level_to_name = ['NONE',
                        'INTERNAL_ERROR',
                        'WRONG_ARGUMENTS',
@@ -62,6 +63,7 @@ error_level_to_name = ['NONE',
                        'WRONG_INSTALL',
                        'RENDER_3D_ERR',
                        'FAILED_EXECUTE',
+                       'KICOST_ERROR',
                        ]
 CMD_EESCHEMA_DO = 'eeschema_do'
 URL_EESCHEMA_DO = 'https://github.com/INTI-CMNB/KiAuto'

kibot/out_bom.py

@@ -237,9 +237,63 @@ class BoMXLSX(BoMLinkable):
             """ Head style: modern-blue, modern-green, modern-red and classic """
             self.kicost = False
             """ Enable KiCost worksheet creation """
+            self.kicost_api_enable = Optionable
+            """ [string|list(string)=''] List of KiCost APIs to enable """
+            self.kicost_api_disable = Optionable
+            """ [string|list(string)=''] List of KiCost APIs to disable """
+            self.kicost_dist_desc = False
+            """ Used to add a column with the distributor's description. So you can check this is the right component """
+            self.kicost_config = ''
+            """ KiCost configuration file. It contains the keys for the different distributors APIs.
+                The regular KiCost config is used when empty """
+            self.specs = False
+            """ Enable Specs worksheet creation. Contains specifications for the components.
+                Works with only some KiCost APIs """
+            self.specs_columns = BoMColumns
+            """ [list(dict)|list(string)] Which columns are included in the Specs worksheet. Use `References` for the references,
+                'Row' for the order and 'Sep' to separate groups at the same level. By default all are included.
+                Column names are distributor specific, the following aren't: '_desc', '_value', '_tolerance', '_footprint',
+                '_power', '_current', '_voltage', '_frequency', '_temp_coeff', '_manf', '_size' """
             self.logo_scale = 2
             """ Scaling factor for the logo. Note that this value isn't honored by all spreadsheet software """
 
+    def process_columns_config(self, cols):
+        if isinstance(cols, type):
+            return (None, None, None, None, None)
+        columns = []
+        column_levels = []
+        column_comments = []
+        column_rename = {}
+        join = []
+        # Create the different lists
+        for col in cols:
+            if isinstance(col, str):
+                # Just a string, add to the list of used
+                new_col = col
+                new_col_l = new_col.lower()
+                level = 0
+                comment = ''
+                if new_col_l[0] == '_':
+                    column_rename[new_col_l] = new_col_l[1:].capitalize()
+            else:
+                # A complete entry
+                new_col = col.field
+                new_col_l = new_col.lower()
+                # A column rename
+                if col.name:
+                    column_rename[new_col_l] = col.name
+                elif new_col_l[0] == '_':
+                    column_rename[new_col_l] = new_col_l[1:].capitalize()
+                # Attach other columns
+                if col.join:
+                    join.append(col.join)
+                level = col.level
+                comment = col.comment
+            columns.append(new_col)
+            column_levels.append(level)
+            column_comments.append(comment)
+        return (columns, column_levels, column_comments, column_rename, join)
+
     def config(self, parent):
         super().config(parent)
         # Style
@@ -247,6 +301,22 @@ class BoMXLSX(BoMLinkable):
             self.style = 'modern-blue'
         if self.style not in VALID_STYLES:
             raise KiPlotConfigurationError('Unknown style `{}`'.format(self.style))
+        if self.kicost_config and not os.path.isfile(self.kicost_config):
+            raise KiPlotConfigurationError('Missing KiCost configuration file `{}`'.format(self.kicost_config))
+        if not self.kicost_config:
+            self.kicost_config = None
+        # KiCost APIs
+        if isinstance(self.kicost_api_enable, type):
+            self.kicost_api_enable = []
+        elif isinstance(self.kicost_api_enable, str):
+            self.kicost_api_enable = [self.kicost_api_enable]
+        if isinstance(self.kicost_api_disable, type):
+            self.kicost_api_disable = []
+        elif isinstance(self.kicost_api_disable, str):
+            self.kicost_api_disable = [self.kicost_api_disable]
+        # Specs columns
+        (self.s_columns, self.s_levels, self.s_comments, self.s_rename,
+         self.s_join) = self.process_columns_config(self.specs_columns)
 
 
 class ComponentAliases(Optionable):

submodules/KiCost

@@ -1 +1 @@
-Subproject commit 28488c34332cdaf83aa159b387a8b75b22af3f58
+Subproject commit cb8747a2b47c6d7074a183b343758460de41d25e

tests/data/.gitignore

@@ -0,0 +1 @@
+discard

tests/data/kicost_default_config.yaml

@@ -0,0 +1,76 @@
+# KiCost configuration file
+kicost:
+  version: 1
+  # Cache Time To Live in days, -1 is forever
+  # Default is 7
+  cache_ttl: 0
+  # Base directory for the APIs caches
+  # cache_path: ~/.cache/kicost
+
+
+APIs:
+  Digi-Key:
+    # Digi-Key Client ID for a registered APP
+    # client_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+    # Digi-Key Client Secret for a registered APP
+    # client_secret: XXXXXXXXXXXXXXXX
+    # Use the sandbox server, doesn't count the usage, but returns old data
+    # sandbox: false
+    # Only enabled if the client_id and client_secret are defined
+    # enable: true
+    # Directory for the APIs caches
+    # cache_path: ~/.cache/kicost/Digi-Key
+  Mouser:
+    # Mouser Part API key
+    # key: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
+    # Only enabled if the key is defined
+    # enable: false
+    # Directory for the APIs caches
+    # cache_path: ~/.cache/kicost/Mouser
+  Element14:
+    # Element14 includes: Farnell, Newark and CPC
+    # Element14 Product Search API key
+    # key: XXXXXXXXXXXXXXXXXXXXXXXX
+    # Only enabled if the key is defined
+    # enable: false
+    # Country used for Farnell queries.
+    # Supported countries: BG,CZ,DK,AT,CH,DE,IE,IL,UK,ES,EE,FI,FR,HU,IT,LT,
+    # LV,BE,NL,NO,PL,PT,RO,RU,SK,SI,SE,TR,CN,AU,NZ,HK,SG,MY,PH,TH,IN,KR,VN
+    # farnell_country: UK
+    # Country used for Newark queries.
+    # Supported countries: US,CA,MX
+    # newark_country: US
+    # Country used for CPC queries.
+    # Supported countries: UK,IE
+    # cpc_country: UK
+    # Directory for the APIs caches
+    # cache_path: ~/.cache/kicost/Element14
+  Octopart:
+    # Octopart API Key
+    # key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+    # API level: 3 or 4
+    # level: 4
+    # The extended API is for the Pro plan
+    # extended: false
+    # Only enabled if the key is defined
+    # enable: false
+    # Directory for the APIs caches
+    # cache_path: ~/.cache/kicost/Octopart
+  KitSpace:
+    # Normally enabled
+    # enable: true
+    # Directory for the APIs caches
+    cache_path: ./discard
+  TME:
+    # TME token (anonymous or private)
+    # token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+    # TME application secret
+    # app_secret: XXXXXXXXXXXXXXXXXXXX
+    # Only enabled if the token and app_secret are defined
+    # enable: false
+    # Country where we are buying
+    # country: US
+    # Language for the texts
+    # language: EN
+    # Directory for the APIs caches
+    # cache_path: ~/.cache/kicost/TME

tests/test_plot/test_kicost.py

@@ -83,7 +83,7 @@ def test_kicost_bom_sel_dist_1(test_dir):
     """ Internal BoM + KiCost, select distributors (Mouser+Digi-Key). With DNF sheet. """
     prj = 'kibom-variant_2c'
     ctx = context.TestContextSCH(test_dir, 'test_kicost_bom_sel_dist_1', prj, 'int_bom_kicost_sel_dist_1_xlsx', OUT_DIR)
-    ctx.run(kicost=True)  # , extra_debug=True
+    ctx.run(kicost=True, extra_debug=True)  # , extra_debug=True
     output = op.join(OUT_DIR, prj+'-bom.xlsx')
     ctx.expect_out_file(output)
     convert2csv(ctx.get_out_path(output), sheet='Costs')

tests/yaml_samples/int_bom_kicost_merge_xlsx.kibot.yaml

@@ -29,3 +29,4 @@ outputs:
           name: Manufacturer P/N
       xlsx:
         kicost: true
+        kicost_config: tests/data/kicost_default_config.yaml

tests/yaml_samples/int_bom_kicost_merge_xlsx_k6.kibot.yaml

@@ -29,3 +29,4 @@ outputs:
           name: Manufacturer P/N
       xlsx:
         kicost: true
+        kicost_config: tests/data/kicost_default_config.yaml

tests/yaml_samples/int_bom_kicost_sel_dist_1_xlsx.kibot.yaml

@@ -39,3 +39,4 @@ outputs:
         - Digi-Key
       xlsx:
         kicost: true
+        kicost_config: tests/data/kicost_default_config.yaml

tests/yaml_samples/int_bom_kicost_simple_xlsx.kibot.yaml

@@ -36,3 +36,4 @@ outputs:
         - 'Voltage'
       xlsx:
         kicost: true
+        kicost_config: tests/data/kicost_default_config.yaml