CI/CD with KiCad and Gitlab

https://sschueller.github.io/posts/ci-cd-with-kicad-and-gitlab/

by sschueller • 3 years ago

232 65 3 years ago

sschueller.github.io

hkwerf8• 3 years ago

At my previous job I, too, set up CI/CD from Schematics/PCB data to device documentation PDFs, 3D models for mechanical integration, and packages for several of our EMS (electronics manufacturing services, each required different data formats) and other items, such as part database management (EOL checks, availability, prices). I, too, used KiCad and Gitlab, and was honestly quite happy with it. The biggest problem was that at a professional scale, KiCad was crashing a bit more often than one would like and it was tedious to find and fix the bugs or work around them if they were already fixed upstream (which they often were).

Still, this heavily sped up prototyping, as anybody could always fetch the data required to release or produce stuff. Shifting left works for electronics just as well as it does for software. :)

ddoolin• 3 years ago

What are some professional alternatives to KiCad? I've seen Altium (which already appears leagues ahead of KiCad, but the license fee is really high...) but none others.

krasin• 3 years ago

What are some professional alternatives to KiCad? I've seen Altium (which already appears leagues ahead of KiCad, but the license fee is really high...) but none others.

Kicad is on a good track to become as capable as Altium, if we judge over the last 3-4 years of the progress. I made it a habit to give them money annually via the Donate button on their front page: https://www.kicad.org.

hkwerf8• 3 years ago

There's probably no single field where you will find more solutions to the same problem, maybe except for text editors? Common ones would be Mentor Xpedition, PADS, Cadence or OrCAD, though. Another irritatingly common one is Eagle, but last I used it it was quite a bit worse than KiCad.

However, this is a small subset of the actual list. Chances are, there's many companies working with proprietary PCB designers that were discontinued 20 years ago or just maintained for a few companies.

IshKebab• 3 years ago

I think the other two big expensive pro ones are Mentor Expedition and Cadence Allegro. There's also NI Ultiboard.

I haven't used any of them though. I don't think you need them unless you're doing really pro stuff (e.g. a PCIe card or motherboard) or RF design.

On the free side they mostly suck. KiCad is powerful but has AWFUL usability. Don't bother unless you're willing to fight the software and spend hours on tutorials for things that should be trivial. Think early Blender but worse.

The other free options are:

* Eagle: Terrible. Do not bother. No idea why anyone uses this. Doesn't even get copy and paste right.

* gEDA: Awful. Do not use.

* DesignSpark PCB. This is very Windows 95 and a bit odd in some ways, but it's probably the easiest to use software and it's not limited like some free software (frequently there are board size or component count limits). Very decent option. If you're new to PCB design and use Windows I think I would maybe recommend this. Either this or...

* Horizon EDA: A new option that uses the KiCad routing engine but is way more usable. I would say this is the best free option at the moment, and it's open source. Except maybe LibrePCB because...

* LibrePCB: I haven't actually gotten around to trying this yet so I have no idea. May be good, may be awful.

timtzm• 3 years ago

I’m experimenting with flux.ai. Browser based pcb design. It has promise and a good team.

cushychicken9• 3 years ago

How long ago was this?

v6 has been very stable for me.

Early v7 builds were a crap shoot. I may try again later this year but I'm really happy with v6 for now.

hkwerf8• 3 years ago

Most of that work was done 2016 - 2020. Typically we saw issues when people jumped on new features in new releases, so it's pretty much excusable and we'd just forbid the use of these features as long as we didn't really need them.

However, it was not uncommon. I always had a debug build of KiCad on my machine in order to replicate crashes or odd behavior others saw.

It should be noted that we're generally talking about PCBs that could get pretty big and complicated. We certainly uncovered edge cases that were hard to foresee that way.

spyremeown• 3 years ago

Very cool!

Does anybody know something like a translator tool between all the different `yaml`s different CI/CD services use? Or maybe some metalanguage specifically made for that that can run instead of whatever hell dash/bash is.

I wish I could just jump ships altering a `script` tag in whatever `yaml` from whichever service.

pjot• 3 years ago

Take a look at Cue!

https://cuelang.org/

cushychicken9• 3 years ago

Smashing job. I use kibot in a similar fashion locally to generate my fab packages for my kicad designs.

I had it set up as a CI job for a while, but I ended up disabling it for a few reasons, the main one being that I just don't think it's a good idea to constantly create build artifacts for circuit board designs. It has created problems when people have circulated intermediate schematics generated by CI that don't match the hardware we built.

You really only need to archive fab packages (by which I mean schematic, gerbers, BOM, assembly drawing, fab notes, etc) at the points in time where you fab and assemble a board.

Creating more artifacts just creates the possibility for more noise. It's not like software CI as there's not really a way to automate hardware testing in a CI server. Very little incremental value in continuous artifact generation as a result.

sschueller4• 3 years ago

Yes, I was thinking of changing the pipeline to respond to only tags for outputs. Then i can just set a release tag when I am ready to send it to a fab.

cushychicken9• 3 years ago

How many people use the outputs of your pipeline, out of curiosity?

fest• 3 years ago

This is what I'm doing. Though it does break down for smaller projects where I had combined firmware and electronics design in the same repository (not a great idea in retrospect).

teleforce• 3 years ago

I'm curious in what you meant by there's no way to automatically perform hardware testing in CI server?

Is it hardware testing at the simulation/emulation level as a "digital twin" or hardware as testing as in hardware-in-the-loop (HLL)?

flaviut2• 3 years ago

Is it hardware testing at the simulation/emulation level as a "digital twin"

This, very likely.

It's possible to set up simulations of certain sections of the hardware using some Spice software, but it's quite time consuming and I'm not sure if anyone's built a way to compare the output of the simulation to expected output.

Simulating any entire circuit as a whole is impossible, since lots of parts just cannot be simulated.

cushychicken9• 3 years ago

CI servers can't hold scope probes or land on test points.

Brian_K_White• 3 years ago

I pointedly keep gerbers etc .ignored (directory naming convention where gerbers always export to GERBER_${TITLE}_${REVISION} ,and .gitignore includes GERBER_*/)

and only upload gerber zips as release artifacts

I do keep a bom.csv in the repo to link right in the readme. I consider the specific part selection part of the reference not entirely documented in the sch, and so it's source.

Same for .stl and other output meshes from openscad and freecad sources.

Which is kind of a pity because github has a neat stl diff viewr that's essentially useless if you don't want to check in stls.

bb88• 3 years ago

@sschueller

Awesome job.

Can you make a base gitlab project so anyone can fork it easily when anyone wants to create a new project?

cushychicken9• 3 years ago

I'd also recommend submitting this as an example use of kibot to the kibot repo on GitHub.

Kibot is really long on functionality - the maintainer is someone in South America who does a pretty incredible job of maintaining it and integrating commonly used plug-ins.

The one thing it really lacks is a comprehensive starting example of how to use it and structure a YAML file.

This would be a great example.

imrehg• 3 years ago

This is very cool! As person doing small scale PCBs in a previous life, might just try to make things set up with my repo similarly (though have to find it first, might be in GitHub, so the idea would be the same but the tooling somewaht different).

One thing that makes me wonder all the time are CI/CD runs that commit back to the repository. That feels weird for some reason I cannot articulate very well yet, but makes me think there should be a better way. Especially if the results are artefacts like that. Any experience by others to weigh in?

brightball• 3 years ago

Depends on the setup and branching execution rules I would think.

stefanoco2• 3 years ago

Great!!! A big part of the value here is the clear and complete explanation, with pointers to external references. May I ask you how do you perform the part: “All these artifacts are commited back into the master and a version is applied. I also have a CHANGELOG.md which is automatically filled from commit messages (I used conventional commits to format my commit messages)”

stefanoco2• 3 years ago

…found myself, I didn’t know about the full capabilities of semantic-release cited in the post!

iamflimflam1• 3 years ago

Really nice. Should be reasonably easy to turn into a GitHub workflow.

Do you have any write ups on the blender rendering and animation? If that could be automated I know a lot of people who would love to use it.

sschueller4• 3 years ago

I will make a write up for that as well when I get a chance.

nimish• 3 years ago

I've ripped out chunks of KiCad to run as CI/CD on the basic PCBs I have. KiCad's data and code is really not geared up for this, so quite a bit had to be hacked up to make it idempotent and even just to bring it up to basic C++ coding standards.

I've been looking at upstreaming what I can and what would even make sense. Ultimately I've switched to mostly using non-KiCad python (dealing with the messy nature of the bundled python wasted 3 solid days, don't use it unless you must) to generate and manage a parts DB + vector drawings that are then processed into the KiCad formats.

It works, but a lot of KiCad internals were simply never designed to handle this scenario. Even multiple PCBs are a pain. It uses multiple proprietary vector representations, some for rendering and some for analysis, etc. The file formats themselves are ok to deal with but aren't well-specified so they will change on you.

Fortunately, the end Gerbers are well defined and it's often easier to simply generate those directly vs going through KiCad if the DRC are guaranteed by construction to never be violated

stavros• 3 years ago

I have done something similar, and have also successfully used the outputs with JLCPCB's assembly service, if you want something similar that's been tested and works:

https://gitlab.com/stavros/sensor-board/

tempodox• 3 years ago

The Blender render is the pinnacle. Awesome job.

junon2• 3 years ago

Side note: where do people get their high-res, beautiful STEP models for renders? I use componentsearchengine but it's rare they have decent models for anything other than the primitive components.

sschueller4• 3 years ago

I use what I can find on step files. Grabcad sometimes or directly from the component maker.

The render on my page is however a bit more work. I graft textures over the parts and set material properties such as reflections. I will do a write up on it when I get a chance.

junon2• 3 years ago

Okay so you're doing some extra stuff in Blender? Makes sense! Appreciate the response, it's beautiful work :)

RF_Savage3• 3 years ago

Manufacturer provided models, 3dcontentcentral and grabcad for me. In that sequence. Altiums built-in step model generator is also quite decent.

Snapeda and componentsearchengine are also getting better.

tuna-f1sh• 3 years ago

I found KiBot configuration a bit overwhelming and with the addition of the ‘kicad-cli’ tool in KiCad 7 made a Makefile based runner: https://github.com/tuna-f1sh/kicad-makefile

It allows for a build that is closer to the vanilla KiCad GUI experience and can be built locally with the normal install using ‘make’ - something I wanted.

CI/CD with KiCad and Gitlab

Related Stories