templates
/
KiBot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Added dependency collection mechanism 

- Currently used to better document the dependencies
This commit is contained in:
Salvador E. Tropea 2022-05-19 08:47:06 -03:00
parent 23e632303d
commit ca861fc21e
35 changed files with 415 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

80
README.md
View File

@ -90,21 +90,65 @@ You can also run KiBot using docker images in a CI/CD environment like GitHub or
### Dependencies
- For ERC, DRC, BoM XML update and SCH print install [KiCad Automation Scripts](https://github.com/INTI-CMNB/kicad-automation-scripts/)
- BoM files (HTML/CSV/TSV/TXT/XML/XLSX) can be generated using the internal BoM generator or using [KiBoM](https://github.com/INTI-CMNB/KiBoM).
- For interactive BoM install [InteractiveHtmlBom](https://github.com/INTI-CMNB/InteractiveHtmlBom)
- For SVG/PNG/JPG beauty PCB render [PcbDraw](https://github.com/INTI-CMNB/PcbDraw). Also install the convert (from imagemagick) and rsvg-convert (from librsvg2-bin) tools.
- For BoMs with costs information install [KiCost](https://github.com/hildogjr/KiCost/)
- To create RAR files install the rar tool.
- To generate reports in PDF format (also ODF, DOCX, etc.) install [Pandoc](https://pandoc.org/)
- If you need to generate the PCB prints in postscript format install ghostscript
- The following Python modules are also used: (Note that using pip or the Debian package they will be installed automatically)
- `colorama`
- `distutils`. This is part of Python, but on debian systems this is in a separated package: `python3-distutils`
- `qrcodegen`. Only to generate QR code symbols and footprints.
- `requests`
- `xlsxwriter`. If you need BoMs in XLSX format.
- `yaml`
Notes:
- When installing from the Debian repo you don't need to worry about dependencies, just pay attention to *recommended* and *suggested* packages.
- When installing using `pip` the dependencies marked as **PyPi dependency** will be automatically installed.
[**distutils**](https://pypi.org/project/distutils/) (python module) [Debian](https://packages.debian.org/bullseye/python3-distutils)
- Mandatory
[**PyYAML**](https://pypi.org/project/PyYAML/) (python module) (PyPi dependency) [Debian](https://packages.debian.org/bullseye/python3-yaml)
- Mandatory
[**requests**](https://pypi.org/project/requests/) (python module) (PyPi dependency) [Debian](https://packages.debian.org/bullseye/python3-requests)
- Mandatory
[**KiCad Automation tools**](https://github.com/INTI-CMNB/KiAuto) v1.6.11 (tool) (PyPi dependency)
- Mandatory for: `gencad`, `netlist`, `pdf_pcb_print`, `pdf_sch_print`, `render_3d`, `run_drc`, `run_erc`, `step`, `svg_pcb_print`, `svg_sch_print`, `update_xml`
[**KiCost**](https://github.com/INTI-CMNB/KiCost) v1.1.8 (tool)
- Mandatory for `kicost`
- Optional to find components costs and specs for `bom`
[**Interactive HTML BoM**](https://github.com/INTI-CMNB/InteractiveHtmlBom) v2.4.1.3 (tool)
- Mandatory for `ibom`
[**KiBoM**](https://github.com/INTI-CMNB/KiBoM) v1.8.0 (tool)
- Mandatory for `kibom`
[**lxml**](https://pypi.org/project/lxml/) (python module) [Debian](https://packages.debian.org/bullseye/python3-lxml)
- Mandatory for `pcb_print`
[**PcbDraw**](https://github.com/INTI-CMNB/pcbdraw) v0.9.0 (tool)
- Mandatory for `pcbdraw`
[**qrcodegen**](https://pypi.org/project/qrcodegen/) (python module) (PyPi dependency) [Debian](https://packages.debian.org/bullseye/python3-qrcodegen)
- Mandatory for `qr_lib`
[**colorama**](https://pypi.org/project/colorama/) (python module) (PyPi dependency) [Debian](https://packages.debian.org/bullseye/python3-colorama)
- Optional to get color messages in a portable way for general use
[**RSVG tools**](https://cran.r-project.org/web/packages/rsvg/index.html) (tool) [Debian](https://packages.debian.org/bullseye/librsvg2-bin)
- Optional to:
- Create PDF, PNG, EPS and PS formats for `pcb_print`
- Create PNG and JPG images for `pcbdraw`
[**Ghostscript**](https://www.ghostscript.com/) (tool) [Debian](https://packages.debian.org/bullseye/ghostscript)
- Optional to create PS files for `pcb_print`
[**ImageMagick**](https://imagemagick.org/) (tool) [Debian](https://packages.debian.org/bullseye/imagemagick)
- Optional to create JPG images for `pcbdraw`
[**Pandoc**](https://pandoc.org/) (tool) [Debian](https://packages.debian.org/bullseye/pandoc)
- Optional to create PDF/ODF/DOCX files for `report`
[**RAR**](https://www.rarlab.com/) (tool) [Debian](https://packages.debian.org/bullseye/rar)
- Optional to compress in RAR format for `compress`
[**xlsxwriter**](https://pypi.org/project/xlsxwriter/) (python module) (PyPi dependency) [Debian](https://packages.debian.org/bullseye/python3-xlsxwriter)
- Optional to create XLSX files for `bom`
### Installation on Ubuntu/Debian
@ -1622,7 +1666,7 @@ Next time you need this list just use an alias, like this:
- `dark_mode`: [boolean=false] Default to dark mode.
- `dnf_filter`: [string|list(string)='_none'] Name of the filter to mark components as not fitted.
A short-cut to use for simple cases where a variant is an overkill.
Avoid using it in conjunction with with IBoM native filtering options.
Avoid using it in conjunction with IBoM native filtering options.
- `dnp_field`: [string=''] Name of the extra field that indicates do not populate status.
Components with this field not empty will be blacklisted.
IBoM option, avoid using in conjunction with KiBot variants/filters.
@ -1660,7 +1704,7 @@ Next time you need this list just use an alias, like this:
Value and Footprint are displayed when nothing is specified.
- `sort_order`: [string='C,R,L,D,U,Y,X,F,SW,A,~,HS,CNN,J,P,NT,MH'] Default sort order for components. Must contain '~' once.
- `variant`: [string=''] Board variant to apply.
Avoid using it in conjunction with with IBoM native filtering options.
Avoid using it in conjunction with IBoM native filtering options.
- `variant_field`: [string=''] Name of the extra field that stores board variant for component.
IBoM option, avoid using in conjunction with KiBot variants/filters.
- `variants_blacklist`: [string=''] List of board variants to exclude from the BOM.
@ -2982,6 +3026,7 @@ Usage:
kibot [-v...] [--start PATH] [-d OUT_DIR] [--dry] [-t, --type TYPE]...
--quick-start
kibot [-v...] --help-filters
kibot [-v...] [--markdown] --help-dependencies
kibot [-v...] --help-global-options
kibot [-v...] --help-list-outputs
kibot [-v...] --help-output=HELP_OUTPUT
@ -3019,6 +3064,7 @@ Quick start options:
Help options:
-h, --help Show this help message and exit
--help-dependencies List dependencies in human readable format
--help-filters List supported filters and details
--help-global-options List supported global variables
--help-list-outputs List supported outputs

20
docs/README.in
View File

@ -90,21 +90,11 @@ You can also run KiBot using docker images in a CI/CD environment like GitHub or
### Dependencies
- For ERC, DRC, BoM XML update and SCH print install [KiCad Automation Scripts](https://github.com/INTI-CMNB/kicad-automation-scripts/)
- BoM files (HTML/CSV/TSV/TXT/XML/XLSX) can be generated using the internal BoM generator or using [KiBoM](https://github.com/INTI-CMNB/KiBoM).
- For interactive BoM install [InteractiveHtmlBom](https://github.com/INTI-CMNB/InteractiveHtmlBom)
- For SVG/PNG/JPG beauty PCB render [PcbDraw](https://github.com/INTI-CMNB/PcbDraw). Also install the convert (from imagemagick) and rsvg-convert (from librsvg2-bin) tools.
- For BoMs with costs information install [KiCost](https://github.com/hildogjr/KiCost/)
- To create RAR files install the rar tool.
- To generate reports in PDF format (also ODF, DOCX, etc.) install [Pandoc](https://pandoc.org/)
- If you need to generate the PCB prints in postscript format install ghostscript
- The following Python modules are also used: (Note that using pip or the Debian package they will be installed automatically)
- `colorama`
- `distutils`. This is part of Python, but on debian systems this is in a separated package: `python3-distutils`
- `qrcodegen`. Only to generate QR code symbols and footprints.
- `requests`
- `xlsxwriter`. If you need BoMs in XLSX format.
- `yaml`
Notes:
- When installing from the Debian repo you don't need to worry about dependencies, just pay attention to *recommended* and *suggested* packages.
- When installing using `pip` the dependencies marked as **PyPi dependency** will be automatically installed.
@dependencies@
### Installation on Ubuntu/Debian

2
docs/replace_tags.pl
View File

@ -5,6 +5,7 @@ $cmd_help=`../src/kibot --help`;
$preflight=`../src/kibot --help-preflights`;
$filters=`../src/kibot --help-filters`;
$global_options=`../src/kibot --help-global-options`;
$dependencies=`../src/kibot --help-dependencies --markdown`;
while (<>)
{
@ -13,6 +14,7 @@ while (<>)
$_ =~ s/\@preflight\@/$preflight/;
$_ =~ s/\@filters\@/$filters/;
$_ =~ s/\@global_options\@/$global_options/;
$_ =~ s/\@dependencies\@/$dependencies/;
print $_;
}

4
docs/samples/generic_plot.kibot.yaml
View File

@ -714,7 +714,7 @@ outputs:
dark_mode: false
# [string|list(string)='_none'] Name of the filter to mark components as not fitted.
# A short-cut to use for simple cases where a variant is an overkill.
# Avoid using it in conjunction with with IBoM native filtering options
# Avoid using it in conjunction with IBoM native filtering options
dnf_filter: '_none'
# [string=''] Name of the extra field that indicates do not populate status.
# Components with this field not empty will be blacklisted.
@ -772,7 +772,7 @@ outputs:
# [string='C,R,L,D,U,Y,X,F,SW,A,~,HS,CNN,J,P,NT,MH'] Default sort order for components. Must contain '~' once
sort_order: 'C,R,L,D,U,Y,X,F,SW,A,~,HS,CNN,J,P,NT,MH'
# [string=''] Board variant to apply.
# Avoid using it in conjunction with with IBoM native filtering options
# Avoid using it in conjunction with IBoM native filtering options
variant: ''
# [string=''] Name of the extra field that stores board variant for component.
# IBoM option, avoid using in conjunction with KiBot variants/filters

10
kibot/__init__.py
View File

@ -1 +1,9 @@
# This file indicates this directory is a Python package
__author__ = 'Salvador E. Tropea, John Beard'
__copyright__ = 'Copyright 2018-2022, Salvador E. Tropea/INTI/John Beard'
__credits__ = ['Salvador E. Tropea', 'John Beard']
__license__ = 'GPL v3+'
__email__ = 'stropea@inti.gob.ar'
__url__ = 'https://github.com/INTI-CMNB/KiBot/'
__status__ = 'stable'
__version__ = '1.0.0'
__pypi_deps__ = ['kiauto', 'pyyaml', 'xlsxwriter', 'colorama', 'requests', 'qrcodegen']

19
kibot/__main__.py
View File

@ -15,6 +15,7 @@ Usage:
kibot [-v...] [--start PATH] [-d OUT_DIR] [--dry] [-t, --type TYPE]...
--quick-start
kibot [-v...] --help-filters
kibot [-v...] [--markdown] --help-dependencies
kibot [-v...] --help-global-options
kibot [-v...] --help-list-outputs
kibot [-v...] --help-output=HELP_OUTPUT
@ -52,6 +53,7 @@ Quick start options:
Help options:
-h, --help Show this help message and exit
--help-dependencies List dependencies in human readable format
--help-filters List supported filters and details
--help-global-options List supported global variables
--help-list-outputs List supported outputs
@ -60,16 +62,6 @@ Help options:
--help-preflights List supported preflights and details
"""
__author__ = 'Salvador E. Tropea, John Beard'
__copyright__ = 'Copyright 2018-2022, Salvador E. Tropea/INTI/John Beard'
__credits__ = ['Salvador E. Tropea', 'John Beard']
__license__ = 'GPL v3+'
__email__ = 'stropea@inti.gob.ar'
__url__ = 'https://github.com/INTI-CMNB/KiBot/'
__status__ = 'stable'
__version__ = '1.0.0'
import os
import sys
from sys import path as sys_path
@ -77,7 +69,7 @@ import re
import gzip
import locale
from glob import glob
from . import __version__, __copyright__, __license__
# Import log first to set the domain
from . import log
log.set_domain('kibot')
@ -100,7 +92,7 @@ from .gs import (GS)
from .misc import (EXIT_BAD_ARGS, W_VARCFG, NO_PCBNEW_MODULE, W_NOKIVER, hide_stderr)
from .pre_base import (BasePreFlight)
from .config_reader import (CfgYamlReader, print_outputs_help, print_output_help, print_preflights_help, create_example,
print_filters_help, print_global_options_help)
print_filters_help, print_global_options_help, print_dependencies)
from .kiplot import (generate_outputs, load_actions, config_output, generate_makefile, generate_examples, solve_schematic,
solve_board_file, solve_project_file, check_board_file)
GS.kibot_version = __version__
@ -274,6 +266,9 @@ def main():
if args.help_global_options:
print_global_options_help()
sys.exit(0)
if args.help_dependencies:
print_dependencies(args.markdown)
sys.exit(0)
if args.example:
check_board_file(args.board_file)
if args.copy_options and not args.board_file:

4
kibot/bom/xlsx_writer.py
View File

@ -21,7 +21,7 @@ from .. import log
from ..misc import W_NOKICOST, W_UNKDIST, KICOST_ERROR, W_BADFIELD
from ..error import trace_dump
from ..gs import GS
from ..__main__ import __version__
from .. import __version__
try:
from xlsxwriter import Workbook
XLSX_SUPPORT = True
@ -549,7 +549,7 @@ def dis_enable_apis(api_options, cfg):
def _create_kicost_sheet(workbook, groups, image_data, fmt_title, fmt_info, fmt_subtitle, fmt_head, fmt_cols, cfg):
if not KICOST_SUPPORT:
logger.warning(W_NOKICOST, 'KiCost sheet requested but failed to load KiCost support')
logger.warning(W_NOKICOST+'KiCost sheet requested but failed to load KiCost support')
return
if cfg.debug_level > 2:
logger.debug("Groups exported to KiCost:")

75
kibot/config_reader.py
View File

@ -17,13 +17,18 @@ from .error import (KiPlotConfigurationError, config_error)
from .misc import (NO_YAML_MODULE, EXIT_BAD_ARGS, EXAMPLE_CFG, WONT_OVERWRITE, W_NOOUTPUTS, W_UNKOUT, W_NOFILTERS,
W_NOVARIANTS, W_NOGLOBALS)
from .gs import GS
from .registrable import RegOutput, RegVariant, RegFilter
from .registrable import RegOutput, RegVariant, RegFilter, RegDependency
from .pre_base import BasePreFlight
from . import __pypi_deps__
# Logger
from . import log
logger = log.get_logger()
LOCAL_OPTIONAL = 1
GLOBAL_OPTIONAL = LOCAL_OPTIONAL*100
LOCAL_MANDATORY = GLOBAL_OPTIONAL*100
GLOBAL_MANDATORY = LOCAL_MANDATORY*100
try:
import yaml
@ -683,3 +688,69 @@ def create_example(pcb_file, out_dir, copy_options, copy_expand):
f.write(" description: '{}'\n".format(layer.description))
else:
f.write(' layers: {}\n'.format(layers))
def global2human(name):
return '`'+name+'`' if name != 'global' else 'general use'
def print_dependencies(markdown=True):
# Compute the importance of each dependency
for dep in RegDependency.get_registered().values():
importance = 0
for r in dep.roles:
local = r.output != 'global'
if r.mandatory:
importance += LOCAL_MANDATORY if local else GLOBAL_MANDATORY
else:
importance += LOCAL_OPTIONAL if local else GLOBAL_OPTIONAL
dep.importance = importance
# Now print them sorted by importance (and by name as a second criteria)
for name, dep in sorted(sorted(RegDependency.get_registered().items(), key=lambda x: x[0].lower()), # noqa C414
key=lambda x: x[1].importance, reverse=True):
dtype = 'python module' if dep.is_python else 'tool'
is_pypi_dep = ' (PyPi dependency)' if dep.pypi_name.lower() in __pypi_deps__ else ''
deb = ''
if markdown:
if dep.is_python:
url = 'https://pypi.org/project/{}/'.format(name)
else:
url = dep.url
name = '[**{}**]({})'.format(name, url)
if dep.in_debian:
deb = ' [Debian](https://packages.debian.org/bullseye/{})'.format(dep.deb_package)
needed = []
optional = []
version = None
for r in dep.roles:
if r.mandatory:
needed.append(global2human(r.output))
else:
optional.append(r)
if r.version and (version is None or r.version > version):
version = r.version
ver = ''
if version:
ver = 'v'+'.'.join(map(str, version))+' '
print("{} {}({}){}{}".format(name, ver, dtype, is_pypi_dep, deb))
if needed:
if len(needed) == 1:
if needed[0] == 'general use':
print('- Mandatory')
else:
print('- Mandatory for '+needed[0])
else:
print('- Mandatory for: '+', '.join(sorted(needed)))
if optional:
if len(optional) == 1:
o = optional[0]
desc = o.desc[0].lower()+o.desc[1:]
print('- Optional to {} for {}'.format(desc, global2human(o.output)))
else:
print('- Optional to:')
for o in optional:
ver = ''
if o.version:
ver = ' (v'+'.'.join(map(str, o.version))+')'
print(' - {} for {}{}'.format(o.desc, global2human(o.output), ver))
print()

43
kibot/create_pdf.py Normal file
View File

@ -0,0 +1,43 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2022 Salvador E. Tropea
# Copyright (c) 2022 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2022 Albin Dennevi (create_pdf_from_pages)
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
# Base idea: https://gitlab.com/dennevi/Board2Pdf/ (Released as Public Domain)
from . import PyPDF2
from .error import KiPlotConfigurationError
def create_pdf_from_pages(input_files, output_fn):
output = PyPDF2.PdfFileWriter()
# Collect all pages
open_files = []
er = None
for filename in input_files:
try:
file = open(filename, 'rb')
open_files.append(file)
pdf_reader = PyPDF2.PdfFileReader(file)
page_obj = pdf_reader.getPage(0)
page_obj.compressContentStreams()
output.addPage(page_obj)
except (IOError, ValueError, EOFError) as e:
er = str(e)
if er:
raise KiPlotConfigurationError('Error reading `{}` ({})'.format(filename, er))
# Write all pages to a file
pdf_output = None
try:
pdf_output = open(output_fn, 'wb')
output.write(pdf_output)
except (IOError, ValueError, EOFError) as e:
er = str(e)
finally:
if pdf_output:
pdf_output.close()
if er:
raise KiPlotConfigurationError('Error creating `{}` ({})'.format(output_fn, er))
# Close the files
for f in open_files:
f.close()

60
kibot/misc.py
View File

@ -293,3 +293,63 @@ def hide_stderr():
os.close(devnull)
yield
os.dup2(newstderr, 2)
class ToolDependencyRole(object):
""" Class used to define the role of a tool """
def __init__(self, desc=None, version=None, output=None):
# Is this tool mandatory
self.mandatory = desc is None
# If not mandatory, for what?
self.desc = desc
# Which version is needed?
self.version = version
# Which output needs it?
self.output = output
class ToolDependency(object):
""" Class used to define tools needed for an output """
def __init__(self, output, name, url=None, url_down=None, is_python=False, deb=None, in_debian=True, extra_deb=None,
roles=None, is_kicad_plugin=False, command=None, pypi_name=None):
# The associated output
self.output = output
# Name of the tool
self.name = name
# Name of the .deb
if deb is None:
if is_python:
self.deb_package = 'python3-'+name.lower()
else:
self.deb_package = name.lower()
else:
self.deb_package = deb
self.is_python = is_python
# If this tool has an official Debian package
self.in_debian = in_debian
# Name at PyPi, can be fake for things that aren't at PyPi
# Is used just to indicate if a dependency will we installed from PyPi
self.pypi_name = pypi_name if pypi_name is not None else name
# Extra Debian packages needed to complement it
self.extra_deb = extra_deb
# URLs
self.url = url
self.url_down = url_down
# Can be installed as a KiCad plug-in?
self.is_kicad_plugin = is_kicad_plugin
# Command we run
self.command = command
# Roles
if roles is None:
roles = [ToolDependencyRole()]
elif not isinstance(roles, list):
roles = [roles]
for r in roles:
r.output = output
self.roles = roles
def kiauto_dependency(output, version=None):
role = None if version is None else ToolDependencyRole(version=version)
return ToolDependency(output, 'KiCad Automation tools', URL_EESCHEMA_DO, url_down=URL_EESCHEMA_DO+'/releases',
in_debian=False, pypi_name='kiauto', roles=role)

7
kibot/out_any_pcb_print.py
View File

@ -9,8 +9,9 @@ from .pre_base import BasePreFlight
from .error import KiPlotConfigurationError
from .gs import GS
from .kiplot import check_script, exec_with_retry, add_extra_options
from .misc import CMD_PCBNEW_PRINT_LAYERS, URL_PCBNEW_PRINT_LAYERS, PDF_PCB_PRINT
from .misc import CMD_PCBNEW_PRINT_LAYERS, URL_PCBNEW_PRINT_LAYERS, PDF_PCB_PRINT, kiauto_dependency
from .out_base import VariantOptions
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from .layer import Layer
from . import log
@ -18,6 +19,10 @@ from . import log
logger = log.get_logger()
def register_deps(pre):
RegDependency.register(kiauto_dependency(pre+'_pcb_print', (1, 6, 7)))
class Any_PCB_PrintOptions(VariantOptions):
# Mappings to KiCad config values. They should be the same used in drill_marks.py
_drill_marks_map = {'none': 0, 'small': 1, 'full': 2}

7
kibot/out_any_sch_print.py
View File

@ -8,15 +8,20 @@ from tempfile import mkdtemp
from shutil import rmtree, copy2
from .gs import GS
from .kiplot import check_eeschema_do, exec_with_retry, add_extra_options
from .misc import CMD_EESCHEMA_DO
from .misc import CMD_EESCHEMA_DO, kiauto_dependency
from .out_base import VariantOptions
from .kicad.config import KiConf
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
def register_deps(pre):
RegDependency.register(kiauto_dependency(pre+'_sch_print'))
def copy_project(sch_dir):
""" Copy the project file to the temporal dir """
ext = GS.pro_ext

8
kibot/out_bom.py
View File

@ -11,9 +11,9 @@ import os
import re
from copy import deepcopy
from .gs import GS
from .misc import W_BADFIELD, W_NEEDSPCB, DISTRIBUTORS
from .misc import W_BADFIELD, W_NEEDSPCB, DISTRIBUTORS, ToolDependency, ToolDependencyRole, URL_KICOST
from .optionable import Optionable, BaseOptions
from .registrable import RegOutput
from .registrable import RegOutput, RegDependency
from .error import KiPlotConfigurationError
from .kiplot import get_board_comps_data, load_any_sch
from .bom.columnlist import ColumnList, BoMError
@ -37,6 +37,10 @@ DEFAULT_ALIASES = [['r', 'r_small', 'res', 'resistor'],
['zener', 'zenersmall'],
['d', 'diode', 'd_small'],
]
RegDependency.register(ToolDependency('bom', 'KiCost', URL_KICOST, url_down=URL_KICOST+'/releases', in_debian=False,
roles=ToolDependencyRole(desc='Find components costs and specs', version=(1, 1, 8))))
RegDependency.register(ToolDependency('bom', 'xlsxwriter', is_python=True,
roles=ToolDependencyRole(desc='Create XLSX files')))
class BoMJoinField(Optionable):

7
kibot/out_compress.py
View File

@ -14,13 +14,16 @@ from tarfile import open as tar_open
from collections import OrderedDict
from .gs import GS
from .kiplot import config_output, get_output_dir, run_output
from .misc import MISSING_TOOL, WRONG_INSTALL, W_EMPTYZIP, WRONG_ARGUMENTS, INTERNAL_ERROR
from .misc import MISSING_TOOL, WRONG_INSTALL, W_EMPTYZIP, WRONG_ARGUMENTS, INTERNAL_ERROR, ToolDependency, ToolDependencyRole
from .optionable import Optionable, BaseOptions
from .registrable import RegOutput
from .registrable import RegOutput, RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
RegDependency.register(ToolDependency('compress', 'RAR', 'https://www.rarlab.com/',
url_down='https://www.rarlab.com/download.htm',
roles=ToolDependencyRole(desc='Compress in RAR format')))
class FilesList(Optionable):

4
kibot/out_gencad.py
View File

@ -6,12 +6,14 @@
import os
from .gs import GS
from .optionable import BaseOptions
from .misc import CMD_PCBNEW_GENCAD, URL_PCBNEW_GENCAD, FAILED_EXECUTE
from .misc import CMD_PCBNEW_GENCAD, URL_PCBNEW_GENCAD, FAILED_EXECUTE, kiauto_dependency
from .kiplot import check_script, exec_with_retry, add_extra_options
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
RegDependency.register(kiauto_dependency('gencad', (1, 6, 5)))
class GenCADOptions(BaseOptions):

8
kibot/out_ibom.py
View File

@ -5,15 +5,19 @@
# Project: KiBot (formerly KiPlot)
import os
from subprocess import (check_output, STDOUT, CalledProcessError)
from .misc import (CMD_IBOM, URL_IBOM, BOM_ERROR, W_EXTNAME, W_NONETLIST)
from .misc import (CMD_IBOM, URL_IBOM, BOM_ERROR, W_EXTNAME, ToolDependency, ToolDependencyRole, W_NONETLIST)
from .gs import (GS)
from .kiplot import check_script, search_as_plugin
from .out_base import VariantOptions
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
WARNING_MIX = "Avoid using it in conjunction with with IBoM native filtering options"
WARNING_MIX = "Avoid using it in conjunction with IBoM native filtering options"
RegDependency.register(ToolDependency('ibom', 'Interactive HTML BoM', URL_IBOM, url_down=URL_IBOM+'/releases',
is_kicad_plugin=True, command=CMD_IBOM, in_debian=False,
roles=ToolDependencyRole(version=(2, 4, 1, 3))))
def check_tool():

10
kibot/out_kibom.py
View File

@ -1,24 +1,26 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2020-2021 Salvador E. Tropea
# Copyright (c) 2020-2021 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2020-2022 Salvador E. Tropea
# Copyright (c) 2020-2022 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import os
from re import search
from tempfile import NamedTemporaryFile
from subprocess import (check_output, STDOUT, CalledProcessError)
from .misc import (CMD_KIBOM, URL_KIBOM, BOM_ERROR, W_EXTNAME)
from .misc import (CMD_KIBOM, URL_KIBOM, BOM_ERROR, ToolDependency, ToolDependencyRole, W_EXTNAME)
from .kiplot import (check_script)
from .gs import (GS)
from .optionable import Optionable, BaseOptions
from .error import KiPlotConfigurationError
from .bom.columnlist import ColumnList
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
CONFIG_FILENAME = 'config.kibom.ini'
RegDependency.register(ToolDependency('kibom', 'KiBoM', URL_KIBOM, url_down=URL_KIBOM+'/releases',
command=CMD_KIBOM, in_debian=False, roles=ToolDependencyRole(version=(1, 8, 0))))
class KiBoMRegex(Optionable):

9
kibot/out_kicost.py
View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2021 Salvador E. Tropea
# Copyright (c) 2021 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2021-2022 Salvador E. Tropea
# Copyright (c) 2021-2022 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import os
@ -9,7 +9,8 @@ from subprocess import check_output, STDOUT, CalledProcessError
from tempfile import mkdtemp
from shutil import rmtree
from .misc import (CMD_KICOST, URL_KICOST, BOM_ERROR, DISTRIBUTORS, W_UNKDIST, ISO_CURRENCIES, W_UNKCUR, KICOST_SUBMODULE,
W_KICOSTFLD, W_MIXVARIANT)
W_KICOSTFLD, W_MIXVARIANT, ToolDependency, ToolDependencyRole)
from .registrable import RegDependency
from .error import KiPlotConfigurationError
from .optionable import Optionable
from .gs import GS
@ -21,6 +22,8 @@ from . import log
logger = log.get_logger()
WARNING_MIX = ("Don't use the `kicost_variant` when using internal variants/filters")
RegDependency.register(ToolDependency('kicost', 'KiCost', URL_KICOST, url_down=URL_KICOST+'/releases', in_debian=False,
roles=ToolDependencyRole(version=(1, 1, 7))))
class Aggregate(Optionable):

5
kibot/out_netlist.py
View File

@ -6,12 +6,15 @@
import os
from .gs import GS
from .optionable import BaseOptions
from .misc import CMD_PCBNEW_IPC_NETLIST, URL_PCBNEW_IPC_NETLIST, CMD_EESCHEMA_DO, URL_EESCHEMA_DO, FAILED_EXECUTE
from .misc import (CMD_PCBNEW_IPC_NETLIST, URL_PCBNEW_IPC_NETLIST, CMD_EESCHEMA_DO, URL_EESCHEMA_DO, FAILED_EXECUTE,
kiauto_dependency)
from .kiplot import check_script, exec_with_retry, add_extra_options
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
RegDependency.register(kiauto_dependency('netlist', (1, 6, 11)))
class NetlistOptions(BaseOptions):

48
kibot/out_pcb_print.py
View File

@ -23,12 +23,13 @@ from .kicad.config import KiConf
from .kicad.v5_sch import SchError
from .kicad.pcb import PCB
from .misc import (CMD_PCBNEW_PRINT_LAYERS, URL_PCBNEW_PRINT_LAYERS, PDF_PCB_PRINT, MISSING_TOOL, W_PDMASKFAIL,
KICAD5_SVG_SCALE, W_MISSTOOL)
KICAD5_SVG_SCALE, W_MISSTOOL, ToolDependency, ToolDependencyRole)
from .kiplot import check_script, exec_with_retry, add_extra_options
from .registrable import RegDependency
from .create_pdf import create_pdf_from_pages
from .macros import macros, document, output_class # noqa: F401
from .layer import Layer, get_priority
from .__main__ import __version__
from . import PyPDF2
from . import __version__
from . import log
logger = log.get_logger()
@ -41,6 +42,13 @@ POLY_FILL_STYLE = ("fill:{0}; fill-opacity:1.0; stroke:{0}; stroke-width:1; stro
"stroke-linejoin:round;fill-rule:evenodd;")
DRAWING_LAYERS = ['Dwgs.User', 'Cmts.User', 'Eco1.User', 'Eco2.User']
EXTRA_LAYERS = ['F.Fab', 'B.Fab', 'F.CrtYd', 'B.CrtYd']
RegDependency.register(ToolDependency('pcb_print', 'RSVG tools',
'https://cran.r-project.org/web/packages/rsvg/index.html', deb='librsvg2-bin',
roles=ToolDependencyRole(desc='Create PDF, PNG, EPS and PS formats')))
RegDependency.register(ToolDependency('pcb_print', 'Ghostscript', 'https://www.ghostscript.com/',
url_down='https://github.com/ArtifexSoftware/ghostpdl-downloads/releases',
roles=ToolDependencyRole(desc='Create PS files')))
RegDependency.register(ToolDependency('pcb_print', 'lxml', is_python=True))
def _run_command(cmd):
@ -102,40 +110,6 @@ def get_size(svg):
return float(view_box[2]), float(view_box[3])
def create_pdf_from_pages(input_files, output_fn):
output = PyPDF2.PdfFileWriter()
# Collect all pages
open_files = []
er = None
for filename in input_files:
try:
file = open(filename, 'rb')
open_files.append(file)
pdf_reader = PyPDF2.PdfFileReader(file)
page_obj = pdf_reader.getPage(0)
page_obj.compressContentStreams()
output.addPage(page_obj)
except (IOError, ValueError, EOFError) as e:
er = str(e)
if er:
raise KiPlotConfigurationError('Error reading `{}` ({})'.format(filename, er))
# Write all pages to a file
pdf_output = None
try:
pdf_output = open(output_fn, 'wb')
output.write(pdf_output)
except (IOError, ValueError, EOFError) as e:
er = str(e)
finally:
if pdf_output:
pdf_output.close()
if er:
raise KiPlotConfigurationError('Error creating `{}` ({})'.format(output_fn, er))
# Close the files
for f in open_files:
f.close()
def svg_to_pdf(input_folder, svg_file, pdf_file):
# Note: rsvg-convert uses 90 dpi but KiCad (and the docs I found) says SVG pt is 72 dpi
cmd = [SVG2PDF, '-d', '72', '-p', '72', '-f', 'pdf', '-o', os.path.join(input_folder, pdf_file),

14
kibot/out_pcbdraw.py
View File

@ -4,14 +4,15 @@
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import os
from tempfile import (NamedTemporaryFile)
from tempfile import NamedTemporaryFile
# Here we import the whole module to make monkeypatch work
import subprocess
import shutil
from .misc import (PCBDRAW, PCBDRAW_ERR, URL_PCBDRAW, W_AMBLIST, W_UNRETOOL, W_USESVG2, W_USEIMAGICK, PCB_MAT_COLORS,
PCB_FINISH_COLORS, SOLDER_COLORS, SILK_COLORS)
PCB_FINISH_COLORS, SOLDER_COLORS, SILK_COLORS, ToolDependency, ToolDependencyRole)
from .kiplot import check_script
from .gs import (GS)
from .registrable import RegDependency
from .gs import GS
from .optionable import Optionable
from .out_base import VariantOptions
from .macros import macros, document, output_class # noqa: F401
@ -22,6 +23,13 @@ SVG2PNG = 'rsvg-convert'
CONVERT = 'convert'
# 0.9.0 implements KiCad 6 support
MIN_VERSION = '0.9.0'
RegDependency.register(ToolDependency('pcbdraw', 'RSVG tools', 'https://cran.r-project.org/web/packages/rsvg/index.html',
deb='librsvg2-bin',
roles=ToolDependencyRole(desc='Create PNG and JPG images')))
RegDependency.register(ToolDependency('pcbdraw', 'ImageMagick', 'https://imagemagick.org/',
roles=ToolDependencyRole(desc='Create JPG images')))
RegDependency.register(ToolDependency('pcbdraw', 'PcbDraw', URL_PCBDRAW, url_down=URL_PCBDRAW+'/releases', in_debian=False,
roles=ToolDependencyRole(version=(0, 9, 0))))
class PcbDrawStyle(Optionable):

3
kibot/out_pdf_pcb_print.py
View File

@ -4,13 +4,14 @@
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
from .gs import GS
from .out_any_pcb_print import Any_PCB_PrintOptions
from .out_any_pcb_print import Any_PCB_PrintOptions, register_deps
from .error import KiPlotConfigurationError
from .macros import macros, document, output_class # noqa: F401
from .layer import Layer
from . import log
logger = log.get_logger()
register_deps('pdf')
class PDF_PCB_PrintOptions(Any_PCB_PrintOptions):

3
kibot/out_pdf_sch_print.py
View File

@ -4,12 +4,13 @@
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
from .gs import GS
from .out_any_sch_print import Any_SCH_PrintOptions
from .out_any_sch_print import Any_SCH_PrintOptions, register_deps
from .misc import PDF_SCH_PRINT
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
register_deps('pdf')
class PDF_SCH_PrintOptions(Any_SCH_PrintOptions):

2
kibot/out_pdfunite.py
View File

@ -14,8 +14,8 @@ from .kiplot import config_output, get_output_dir, run_output
from .misc import MISSING_TOOL, WRONG_INSTALL, WRONG_ARGUMENTS, INTERNAL_ERROR, W_NOTPDF
from .optionable import Optionable, BaseOptions
from .registrable import RegOutput
from .create_pdf import create_pdf_from_pages
from .macros import macros, document, output_class # noqa: F401
from .out_pcb_print import create_pdf_from_pages
from . import log
logger = log.get_logger()

3
kibot/out_qr_lib.py
View File

@ -12,6 +12,8 @@ from .error import KiPlotConfigurationError
from .kicad.sexpdata import Symbol, dumps, Sep, load, SExpData, sexp_iter
from .kicad.v6_sch import DrawRectangleV6, PointXY, Stroke, Fill, SchematicFieldV6, FontEffects
from .kiplot import load_board
from .misc import ToolDependency, ToolDependencyRole
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
@ -22,6 +24,7 @@ QR_ECCS = {'low': QrCode.Ecc.LOW,
logger = log.get_logger()
TO_SEPARATE = {'kicad_pcb', 'general', 'title_block', 'layers', 'setup', 'pcbplotparams', 'net_class', 'module',
'kicad_sch', 'lib_symbols', 'symbol', 'sheet', 'sheet_instances', 'symbol_instances'}
RegDependency.register(ToolDependency('qr_lib', 'qrcodegen', is_python=True, roles=ToolDependencyRole()))
def is_symbol(name, sexp):

4
kibot/out_render_3d.py
View File

@ -7,14 +7,16 @@
import os
from shutil import rmtree
from .misc import (CMD_PCBNEW_3D, URL_PCBNEW_3D, RENDER_3D_ERR, PCB_MAT_COLORS, PCB_FINISH_COLORS, SOLDER_COLORS, SILK_COLORS,
KICAD_VERSION_6_0_2, MISSING_TOOL)
KICAD_VERSION_6_0_2, MISSING_TOOL, kiauto_dependency)
from .gs import (GS)
from .kiplot import check_script, exec_with_retry, add_extra_options
from .out_base_3d import Base3DOptions, Base3D
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
RegDependency.register(kiauto_dependency('render_3d', (1, 6, 8)))
class Render3DOptions(Base3DOptions):

8
kibot/out_report.py
View File

@ -11,8 +11,8 @@ from shutil import which
from .gs import GS
from .misc import (UI_SMD, UI_VIRTUAL, MOD_THROUGH_HOLE, MOD_SMD, MOD_EXCLUDE_FROM_POS_FILES, PANDOC, MISSING_TOOL,
FAILED_EXECUTE, W_WRONGEXT, W_WRONGOAR, W_ECCLASST, W_MISSTOOL)
from .registrable import RegOutput
FAILED_EXECUTE, W_WRONGEXT, W_WRONGOAR, W_ECCLASST, W_MISSTOOL, ToolDependency, ToolDependencyRole)
from .registrable import RegOutput, RegDependency
from .out_base import BaseOptions
from .error import KiPlotConfigurationError
from .kiplot import config_output
@ -23,6 +23,10 @@ logger = log.get_logger()
INF = float('inf')
PANDOC_INSTALL = ("In CI/CD environments: the `kicad_auto_test` docker image contains it.\n"
"In Debian/Ubuntu environments: install `pandoc`, `texlive-latex-base` and `texlive-latex-recommended`")
RegDependency.register(ToolDependency('report', 'Pandoc', 'https://pandoc.org/',
url_down='https://github.com/jgm/pandoc/releases',
extra_deb=['texlive-latex-base', 'texlive-latex-recommended'],
roles=ToolDependencyRole(desc='Create PDF/ODF/DOCX files')))
def do_round(v, dig):

4
kibot/out_step.py
View File

@ -9,14 +9,16 @@ import re
from subprocess import (check_output, STDOUT, CalledProcessError)
from shutil import rmtree
from .error import KiPlotConfigurationError
from .misc import KICAD2STEP, KICAD2STEP_ERR, URL_PCBNEW_RUN_DRC
from .misc import KICAD2STEP, KICAD2STEP_ERR, URL_PCBNEW_RUN_DRC, kiauto_dependency
from .gs import (GS)
from .out_base_3d import Base3DOptions, Base3D
from .kiplot import check_script, add_extra_options
from .registrable import RegDependency
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
RegDependency.register(kiauto_dependency('step', (1, 6, 1)))
class STEPOptions(Base3DOptions):

3
kibot/out_svg_pcb_print.py
View File

@ -5,7 +5,7 @@
# Project: KiBot (formerly KiPlot)
import os
from .gs import GS
from .out_any_pcb_print import Any_PCB_PrintOptions
from .out_any_pcb_print import Any_PCB_PrintOptions, register_deps
from .error import KiPlotConfigurationError
from .kicad.patch_svg import patch_svg_file
from .macros import macros, document, output_class # noqa: F401
@ -13,6 +13,7 @@ from .layer import Layer
from . import log
logger = log.get_logger()
register_deps('svg')
class SVG_PCB_PrintOptions(Any_PCB_PrintOptions):

3
kibot/out_svg_sch_print.py
View File

@ -5,12 +5,13 @@
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
from .gs import GS
from .out_any_sch_print import Any_SCH_PrintOptions
from .out_any_sch_print import Any_SCH_PrintOptions, register_deps
from .misc import SVG_SCH_PRINT
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger()
register_deps('svg')
class SVG_SCH_PrintOptions(Any_SCH_PrintOptions):

14
kibot/pre_drc.py
View File

@ -1,19 +1,21 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2020-2021 Salvador E. Tropea
# Copyright (c) 2020-2021 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2020-2022 Salvador E. Tropea
# Copyright (c) 2020-2022 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import os
from sys import (exit)
from sys import exit
from .macros import macros, pre_class # noqa: F401
from .error import (KiPlotConfigurationError)
from .gs import (GS)
from .gs import GS
from .optionable import Optionable
from .kiplot import check_script, exec_with_retry, load_board, add_extra_options
from .misc import (CMD_PCBNEW_RUN_DRC, URL_PCBNEW_RUN_DRC, DRC_ERROR)
from .log import (get_logger)
from .misc import CMD_PCBNEW_RUN_DRC, URL_PCBNEW_RUN_DRC, DRC_ERROR, kiauto_dependency
from .registrable import RegDependency
from .log import get_logger
logger = get_logger(__name__)
RegDependency.register(kiauto_dependency('run_drc'))
@pre_class

16
kibot/pre_erc.py
View File

@ -1,19 +1,21 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2020-2021 Salvador E. Tropea
# Copyright (c) 2020-2021 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2020-2022 Salvador E. Tropea
# Copyright (c) 2020-2022 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import os
from sys import (exit)
from sys import exit
from .macros import macros, pre_class # noqa: F401
from .gs import (GS)
from .gs import GS
from .optionable import Optionable
from .kiplot import check_eeschema_do, exec_with_retry, load_sch, add_extra_options
from .error import (KiPlotConfigurationError)
from .misc import (CMD_EESCHEMA_DO, ERC_ERROR)
from .log import (get_logger)
from .error import KiPlotConfigurationError
from .misc import CMD_EESCHEMA_DO, ERC_ERROR, kiauto_dependency
from .registrable import RegDependency
from .log import get_logger
logger = get_logger(__name__)
RegDependency.register(kiauto_dependency('run_erc'))
@pre_class

16
kibot/pre_update_xml.py
View File

@ -1,18 +1,20 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2020-2021 Salvador E. Tropea
# Copyright (c) 2020-2021 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2020-2022 Salvador E. Tropea
# Copyright (c) 2020-2022 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import os
from sys import (exit)
from sys import exit
from .macros import macros, pre_class # noqa: F401
from .error import (KiPlotConfigurationError)
from .gs import (GS)
from .error import KiPlotConfigurationError
from .gs import GS
from .kiplot import check_eeschema_do, exec_with_retry, add_extra_options
from .misc import (CMD_EESCHEMA_DO, BOM_ERROR)
from .log import (get_logger)
from .misc import CMD_EESCHEMA_DO, BOM_ERROR, kiauto_dependency
from .registrable import RegDependency
from .log import get_logger
logger = get_logger(__name__)
RegDependency.register(kiauto_dependency('update_xml'))
@pre_class

27
kibot/registrable.py
View File

@ -6,6 +6,7 @@
from collections import OrderedDict
from .optionable import Optionable
from .error import KiPlotConfigurationError
from .misc import ToolDependency, ToolDependencyRole
class Registrable(object):
@ -148,3 +149,29 @@ class RegFilter(Optionable, Registrable):
def __init__(self):
super().__init__()
class RegDependency(Registrable):
""" Used to register output tools dependencies """
_registered = {}
def __init__(self):
super().__init__()
@classmethod
def register(cl, aclass):
name = aclass.name
if name in cl._registered:
# Already registered, add the roles
old_reg = cl._registered[name]
old_reg.roles.extend(aclass.roles)
else:
cl._registered[name] = aclass
# Here we register some global dependencies
RegDependency.register(ToolDependency('global', 'colorama', is_python=True,
roles=ToolDependencyRole(desc='get color messages in a portable way')))
RegDependency.register(ToolDependency('global', 'distutils', is_python=True))
RegDependency.register(ToolDependency('global', 'requests', is_python=True))
RegDependency.register(ToolDependency('global', 'PyYAML', is_python=True, deb='python3-yaml'))

4
setup.py
View File

@ -1,7 +1,7 @@
#!/usr/bin/python3
from setuptools import setup, find_packages
# Package meta-data, mostly from the package
from kibot.__main__ import __author__, __email__, __url__, __doc__, __version__
from kibot import __author__, __email__, __url__, __doc__, __version__, __pypi_deps__
# Use the README.md as a long description.
# Note this is also included in the MANIFEST.in
@ -20,7 +20,7 @@ setup(name='kibot',
# Packages are marked using __init__.py
packages=find_packages(),
scripts=['src/kibot', 'src/kiplot'],
install_requires=['kiauto', 'pyyaml', 'xlsxwriter', 'colorama', 'requests', 'qrcodegen'],
install_requires=__pypi_deps__,
include_package_data=True,
classifiers=['Development Status :: 5 - Production/Stable',
'Environment :: Console',