In my previous post, I discussed my reason for creating yet another Python script to convert Kicad schematics into a BoM spreadsheet with pricing info from Digikey. This post is a continuation. Since the first post I am happy to say I have gotten all the goo to work and was able to use the MadeDigikeyBOM.csv to order parts from Digikey. So far I am excited with the results. I find them a terrific time saver (of course, learning python and getting all this to work takes A LOT more time than doing one schematic -> BoM by hand…but YIPPEE! What a learning opportunity…and I plan for plenty more schematics.
The goal of this post is to start a code review MakeDigikeyBoM python project. I’ll cover the block diagram I created to represent the “big picture”.
Thanks to Those That Went Before
I could not have gotten this far this quickly without the prior work of debvisme(xesscorp). He is the author of Kicost – Python that “build(s) cost spreadsheet for a Kicad project.” Kicost is well worth checking out. The GitHub location is here. The code I used the most was the script that scrapes the Digikey pages. I learned a lot about the Kicad XML and various Python techniques from other sections of the code.
The folks who created the Beautiful Soup library for Python. Beautiful Soup is incredibly useful/well done to parse XML files.
The MakeDigikeyBom Python project is located at this GitHub location.
Updated Block Diagram
I like to start by doing a fly by of a block diagram. I had done an initial block diagram in the previous post. The block diagram evolved as the design and implementation evolved. Here is the updated block diagram
Input from a Kicad schematic. The important component Value and PN properties are important to this process.
For example, here is an image of Ladybug Blue’s Schematic with it’s hierarchical sheets:
Each schematic component has a PN field and Value field. The PN field must contain one of the following:
The schematic component D1 (RED_LED) with a manufacturer part number PN = XPEBRD-L1-R250-0061. When the PN is a manufacturer or Digikey part number, the value field is not used:
The component C1 has a Jelly Bean PN = C and Value of .1u:
When MakeDigikeyBoM modifies the <outputFrom_bom2csv>.xml file, the created <modified_outputFrom_bom2csv>.xml file replaces the PN field value to the digikey part number CS21F104XZBCNNNC. This is the part I chose for all .1u capacitors.
The P5 component has None in it’s PN field value:
A component with None in the PN field tells MakeDigikeyBom that this component is not purchased through Digikey. The Value field is ignored.
Use the bom2csv plug-in to create the “BoM” XML. The bom2csv plug-in was discussed in my previous post. The XML branch that is the most important to the DIgikey BoM creation process is the <comps> branch. As the tag suggests, this is where the component info is found.
Here are the <comp> properties for the three types of PN discussed above.
A None <comp>
Here, the PN = None. The Value=BNC, but that isn’t used by the Digikey BoM process.
A Jelly Bean <comp>
In the above, component C1 is a .1u of part type C. The Value = .1u is used by the process.
A Manufacturer or Digikey Part Number
As when the PN=None, the Value field is not used. In this case, the PN number is passed directly into the screen scraping section of the Digikey process.
The first part of the MakeDigikeyBoM Python script takes in the output from bom2csv discussed under “B” above and creates an intermediate file that is then used by another part of the Python script. The intermediate file is in the same XML format as the file created by bom2csv. The properties for <comp> components that use a Jelly Bean PN have been replaced by looking up the PN / Value field in the JellyBeanPartNumbers.csv file with the part number that must be should be either the Digikey or manufacturer part.
For the C1 Jelly Bean <comp> listed above, the PN = C and the Value = .1u. The Python script looks for the row in the JellyBeanPartNumbers.csv file:
in this case, the manufacturer part number = CL21F104ZBCNNNC. The modified bom2csv file replaces the <comp> tag for C1 with this part number:
The Python script creates a URL with the PN as keyword to the Digikey web site. The URL is pretty much what we might use when searching for this part:
url = ‘http://www.digikey.com/scripts/DkSearch/dksus.dll?WT.z_header=search_go&lang=en&keywords=CL21F104ZBCNNNC”
The html is parsed by BeautifulSoup into a fairly easy to access html tree.
There is a whole bunch of Goo in the Python script to scrape through the html and parse it out into the MadeDigikeyBoM.csv file. If all goes well, the MadeDigikeyBoM.csv file is created. For example:
I realize the image is rather small…here is the row for .1u capacitors:
There are 12 .1u capacitors. The price per 1 = $.10, 10 = $.031, 100 = $.0144, and 1000 = $0.0144. Digikey has 79,269 of these capacitors in stock. The link to the Digikey page is included.
The challenge is this part of the script will need to be updated when Digikey updates the web page. This – of course – is true of any web page scraping process.
I’ll start going over the Python modules in the next post. I’ve posted the Python scripts at this GitHub location.
Thanks for reading this far. Please find many things to smile about.