templates
/
KiBot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Added support for ZIP/TAR/RAR generation.

This commit is contained in:
Salvador E. Tropea 2021-01-15 13:33:33 -03:00
parent f9dbaf4619
commit 159e8996e8
16 changed files with 268 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@ -20,6 +20,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- [JLCPCB](https://jlcpcb.com/)
- [P-Ban](https://www.p-ban.com/)
- [PCBWay](https://www.pcbway.com)
- Support for ZIP/TAR/RAR generation.
### Changed
- Now the default output name applies to the DRC and ERC report names.

23
README.md
View File

@ -646,6 +646,26 @@ Next time you need this list just use an alias, like this:
- `style`: [string='modern-blue'] Head style: modern-blue, modern-green, modern-red and classic..
- `title`: [string='KiBot Bill of Materials'] BoM title.
* Archiver (files compressor)
* Type: `compress`
* Description: Generates a compressed file containing output files.
This is used to generate groups of files in compressed file format.
* Valid keys:
- `comment`: [string=''] A comment for documentation purposes.
- `dir`: [string='.'] Output directory for the generated files.
- `name`: [string=''] Used to identify this particular output definition.
- `options`: [dict] Options for the `compress` output.
* Valid keys:
- `compression`: [string='auto'] [auto,stored,deflated,bzip2,lzma] Compression algorithm. Use auto to let KiBot select a suitable one.
- `files`: [list(dict)] Which files will be included.
* Valid keys:
- `dest`: [string=''] Destination directory inside the archive, empty means the same of the file.
- `filter`: [string='.*'] A regular expression that source files must match.
- `source`: [string='*'] File names to add, wildcards allowed. Use ** for recursive match.
Note this pattern is applied to the output dir specified with -d comman line option.
- `format`: [string='ZIP'] [ZIP,TAR,RAR] Output file format.
- `output`: [string='%f-%i%v.%x'] Name for the generated archive (%i=name of the output %x=according to format). Affected by global options.
* DXF (Drawing Exchange Format)
* Type: `dxf`
* Description: Exports the PCB to 2D mechanical EDA tools (like AutoCAD).
@ -1450,7 +1470,8 @@ Options:
- For ERC, DRC, BoM XML update and PCB/SCH print install [KiCad Automation Scripts](https://github.com/INTI-CMNB/kicad-automation-scripts/)
- BoM files (HTML/CSV/TSV/TXT/XML/XLSX) can be generated using the internal BoM generator or using [KiBoM](https://github.com/INTI-CMNB/KiBoM).
- For interactive BoM install [InteractiveHtmlBom](https://github.com/INTI-CMNB/InteractiveHtmlBom)
- For SVG/PNG/JPG beauty PCB render [PcbDraw](https://github.com/INTI-CMNB/PcbDraw)
- For SVG/PNG/JPG beauty PCB render [PcbDraw](https://github.com/INTI-CMNB/PcbDraw). Also install the convert (from imagemagick) and rsvg-convert (from librsvg2-bin) tools.
- To create RAR files install the rar tool.
### Installation on Ubuntu/Debian

2
debian/control vendored
View File

@ -11,7 +11,7 @@ Package: kibot
Architecture: all
Multi-Arch: foreign
Depends: ${misc:Depends}, ${python3:Depends}, python3-yaml, kicad (>= 5.1.0), python3-wxgtk4.0
Recommends: kibom.inti-cmnb (>= 1.8.0), interactivehtmlbom.inti-cmnb, pcbdraw, imagemagick, librsvg2-bin, python3-xlsxwriter
Recommends: kibom.inti-cmnb (>= 1.8.0), interactivehtmlbom.inti-cmnb, pcbdraw, imagemagick, librsvg2-bin, python3-xlsxwriter, rar
Description: KiCad Bot
KiBot is a program which helps you to automate the generation of KiCad
output documents easily, repeatable, and most of all, scriptably.

3
docs/README.in
View File

@ -568,7 +568,8 @@ kibot --list
- For ERC, DRC, BoM XML update and PCB/SCH print install [KiCad Automation Scripts](https://github.com/INTI-CMNB/kicad-automation-scripts/)
- BoM files (HTML/CSV/TSV/TXT/XML/XLSX) can be generated using the internal BoM generator or using [KiBoM](https://github.com/INTI-CMNB/KiBoM).
- For interactive BoM install [InteractiveHtmlBom](https://github.com/INTI-CMNB/InteractiveHtmlBom)
- For SVG/PNG/JPG beauty PCB render [PcbDraw](https://github.com/INTI-CMNB/PcbDraw)
- For SVG/PNG/JPG beauty PCB render [PcbDraw](https://github.com/INTI-CMNB/PcbDraw). Also install the convert (from imagemagick) and rsvg-convert (from librsvg2-bin) tools.
- To create RAR files install the rar tool.
### Installation on Ubuntu/Debian

9
docs/samples/Elecrow.kibot.yaml
View File

@ -46,3 +46,12 @@ outputs:
pth_id: ''
npth_id: '-NPTH'
output: "%f%i.TXT"
- name: Elecrow
comment: ZIP file for Elecrow
type: compress
dir: Elecrow
options:
files:
- source: Elecrow/*
dest: /

9
docs/samples/Elecrow_stencil.kibot.yaml
View File

@ -48,3 +48,12 @@ outputs:
pth_id: ''
npth_id: '-NPTH'
output: "%f%i.TXT"
- name: Elecrow
comment: ZIP file for Elecrow
type: compress
dir: Elecrow
options:
files:
- source: Elecrow/*
dest: /

9
docs/samples/FusionPCB.kibot.yaml
View File

@ -46,3 +46,12 @@ outputs:
pth_and_npth_single_file: true
use_aux_axis_as_origin: true
output: "%f.TXT"
- name: FusionPCB
comment: ZIP file for FusionPCB
type: compress
dir: FusionPCB
options:
files:
- source: FusionPCB/*
dest: /

9
docs/samples/JLCPCB.kibot.yaml
View File

@ -52,3 +52,12 @@ outputs:
npth_id: '-NPTH'
metric_units: false
output: "%f%i.%x"
- name: JLCPCB
comment: ZIP file for JLCPCB
type: compress
dir: JLCPCB
options:
files:
- source: JLCPCB/*
dest: /

9
docs/samples/JLCPCB_stencil.kibot.yaml
View File

@ -45,3 +45,12 @@ outputs:
npth_id: '-NPTH'
metric_units: false
output: "%f%i.%x"
- name: JLCPCB
comment: ZIP file for JLCPCB
type: compress
dir: JLCPCB
options:
files:
- source: JLCPCB/*
dest: /

9
docs/samples/P-Ban.kibot.yaml
View File

@ -59,3 +59,12 @@ outputs:
left_digits: 3
right_digits: 3
output: "%f.drl"
- name: P-Ban
comment: ZIP file for P-Ban
type: compress
dir: P-Ban
options:
files:
- source: P-Ban/*
dest: /

10
docs/samples/PCBWay.kibot.yaml
View File

@ -51,3 +51,13 @@ outputs:
pth_id: ''
npth_id: '-NPTH'
output: "%f%i.drl"
- name: PCBWay
comment: ZIP file for PCBWay
type: compress
dir: PCBWay
options:
format: ZIP
files:
- source: PCBWay/*
dest: /

23
docs/samples/generic_plot.kibot.yaml
View File

@ -161,6 +161,29 @@ outputs:
# [string='KiBot Bill of Materials'] BoM title
title: 'KiBot Bill of Materials'
# Archiver (files compressor):
# This is used to generate groups of files in compressed file format.
- name: 'compress_example'
comment: 'Generates a compressed file containing output files.'
type: 'compress'
dir: 'Example/compress_dir'
options:
# [string='auto'] [auto,stored,deflated,bzip2,lzma] Compression algorithm. Use auto to let KiBot select a suitable one
compression: 'auto'
# [list(dict)] Which files will be included
files:
# [string=''] Destination directory inside the archive, empty means the same of the file
- dest: ''
# [string='.*'] A regular expression that source files must match
filter: '.*'
# [string='*'] File names to add, wildcards allowed. Use ** for recursive match.
# Note this pattern is applied to the output dir specified with -d comman line option
source: '*'
# [string='ZIP'] [ZIP,TAR,RAR] Output file format
format: 'ZIP'
# [string='%f-%i%v.%x'] Name for the generated archive (%i=name of the output %x=according to format). Affected by global options
output: '%f-%i%v.%x'
# DXF (Drawing Exchange Format):
# This output is what you get from the File/Plot menu in pcbnew.
- name: 'dxf_example'

6
kibot/gs.py
View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2020 Salvador E. Tropea
# Copyright (c) 2020 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2020-2021 Salvador E. Tropea
# Copyright (c) 2020-2021 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import os
@ -57,6 +57,8 @@ class GS(object):
pcb_comp = None
# Current variant/s
variant = None
# Name for the output we are generating
current_output = None
# Global defaults
# This is used as default value for classes supporting "output" option
def_global_output = '%f-%i%v.%x'

1
kibot/kiplot.py
View File

@ -266,6 +266,7 @@ def generate_outputs(outputs, target, invert, skip_pre):
load_sch()
config_output(out)
logger.info('- '+str(out))
GS.current_output = out.name
try:
out.run(get_output_dir(out.dir), board)
except PlotError as e:

5
kibot/misc.py
View File

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2020 Salvador E. Tropea
# Copyright (c) 2020 Instituto Nacional de Tecnología Industrial
# Copyright (c) 2020-2021 Salvador E. Tropea
# Copyright (c) 2020-2021 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
""" Miscellaneous definitions """
@ -162,6 +162,7 @@ W_MISFLDNAME = '(W046) '
W_MISS3D = '(W047) '
W_FAILDL = '(W048) '
W_NOLAYER = '(W049) '
W_EMPTYZIP = '(W050) '
class Rect(object):

147
kibot/out_compress.py Normal file
View File

@ -0,0 +1,147 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2021 Salvador E. Tropea
# Copyright (c) 2021 Instituto Nacional de Tecnología Industrial
# License: GPL-3.0
# Project: KiBot (formerly KiPlot)
import re
import os
import glob
from sys import exit
from subprocess import check_output, STDOUT, CalledProcessError
from zipfile import ZipFile, ZIP_STORED, ZIP_DEFLATED, ZIP_BZIP2, ZIP_LZMA
from tarfile import open as tar_open
from collections import OrderedDict
from .gs import GS
from .misc import MISSING_TOOL, WRONG_INSTALL, W_EMPTYZIP
from .optionable import Optionable, BaseOptions
from .macros import macros, document, output_class # noqa: F401
from . import log
logger = log.get_logger(__name__)
class FilesList(Optionable):
def __init__(self):
super().__init__()
with document:
self.source = '*'
""" File names to add, wildcards allowed. Use ** for recursive match.
Note this pattern is applied to the output dir specified with -d comman line option """
self.filter = '.*'
""" A regular expression that source files must match """
self.dest = ''
""" Destination directory inside the archive, empty means the same of the file """
class CompressOptions(BaseOptions):
ZIP_ALGORITHMS = {'auto': ZIP_DEFLATED,
'stored': ZIP_STORED,
'deflated': ZIP_DEFLATED,
'bzip2': ZIP_BZIP2,
'lzma': ZIP_LZMA}
TAR_MODE = {'auto': 'bz2',
'stored': '',
'deflated': 'gz',
'bzip2': 'bz2',
'lzma': 'xz'}
def __init__(self):
with document:
self.output = GS.def_global_output
""" Name for the generated archive (%i=name of the output %x=according to format) """
self.format = 'ZIP'
""" [ZIP,TAR,RAR] Output file format """
self.compression = 'auto'
""" [auto,stored,deflated,bzip2,lzma] Compression algorithm. Use auto to let KiBot select a suitable one """
self.files = FilesList
""" [list(dict)] Which files will be included """
super().__init__()
def config(self):
super().config()
if isinstance(self.files, type):
self.files = []
logger.warning(W_EMPTYZIP+' No files provided, creating an empty archive')
def create_zip(self, output, files):
with ZipFile(output, 'w', compression=self.ZIP_ALGORITHMS[self.compression], compresslevel=9) as zip:
for fname, dest in files.items():
logger.debug('Adding '+fname+' as '+dest)
zip.write(fname, dest)
def create_tar(self, output, files):
with tar_open(output, 'w:'+self.TAR_MODE[self.compression]) as tar:
for fname, dest in files.items():
logger.debug('Adding '+fname+' as '+dest)
tar.add(fname, dest)
def create_rar(self, output, files):
if os.path.isfile(output):
os.remove(output)
for fname, dest in files.items():
logger.debug('Adding '+fname+' as '+dest)
cmd = ['rar', 'a', '-m5', '-ep', '-ap'+os.path.dirname(dest), output, fname]
try:
check_output(cmd, stderr=STDOUT)
except FileNotFoundError:
logger.error('Missing `rar` command, install it')
exit(MISSING_TOOL)
except CalledProcessError as e: # pragma: no cover
logger.error('Failed to invoke rar command, error {}'.format(e.returncode))
if e.output:
logger.debug('Output from command: '+e.output.decode())
exit(WRONG_INSTALL)
def solve_extension(self):
if self.format == 'ZIP':
return 'zip'
if self.format == 'RAR':
return 'rar'
# TAR
ext = 'tar'
sub_ext = self.TAR_MODE[self.compression]
if sub_ext:
ext += '.'+sub_ext
return ext
def run(self, output_dir, board):
# Output file name
logger.debug('output_dir '+output_dir)
logger.debug('GS.out_dir '+GS.out_dir)
output = self.expand_filename(output_dir, self.output, GS.current_output, self.solve_extension())
logger.debug('Collecting files')
output_real = os.path.realpath(output)
# Collect the files
files = OrderedDict()
for f in self.files:
for fname in filter(re.compile(f.filter).match, glob.iglob(os.path.join(GS.out_dir, f.source), recursive=True)):
fname_real = os.path.realpath(fname)
# Avoid including the output
if fname_real == output_real:
continue
# Compute the destination directory inside the archive
dest = fname
if f.dest:
dest = os.path.join(f.dest, os.path.basename(fname))
else:
dest = os.path.relpath(dest, GS.out_dir)
files[fname_real] = dest
logger.debug('Generating `{}` archive'.format(output))
if self.format == 'ZIP':
self.create_zip(output, files)
elif self.format == 'TAR':
self.create_tar(output, files)
elif self.format == 'RAR':
self.create_rar(output, files)
@output_class
class Compress(BaseOutput): # noqa: F821
""" Archiver (files compressor)
Generates a compressed file containing output files.
This is used to generate groups of files in compressed file format. """
def __init__(self):
super().__init__()
with document:
self.options = CompressOptions
""" [dict] Options for the `compress` output """