Setting up the Raspberry Pi for WiFi

This is a very short post to document how to connect a Raspberry Pi to an iPhone’s “Personal Hotspot.”

I’m about to go on a short trip.  I want to take my Raspberry Pi with me.  Currently, the Raspberry Pi is connected to our network using Ethernet.  I want to access using WiFi.  The challenge with that is I need WiFi access when I am not around a wireless access point.

I decide to use my iPhone’s “Personal Hotspot” feature.


I tried two different methods and found Adafruit’s steps for command line configuration of WiFi to work.  The directions walked me through modifying the interfaces.d file:

sudo nano /etc/network/interfaces

~ $ ifconfig wlan0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet  netmask  broadcast

        inet6 fe80::2a5f:78b0:ba56:b15f  prefixlen 64  scopeid 0x20<link>

        ether b8:27:eb:8f:08:40  txqueuelen 1000  (Ethernet)

        RX packets 4  bytes 910 (910.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 35  bytes 6213 (6.0 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I change to my iPhone’s personal hotspot SSID, and YIPPEE! I connect with my Raspberry Pi.

BUT….the Ethernet was gone on reboot.

# interfaces(5) file used by ifup(8) and ifdown(8)


# Please note that this file is written to be used with dhcpcd

# For static IP, consult /etc/dhcpcd.conf and ‘man dhcpcd.conf’


# Include files from /etc/network/interfaces.d:

source-directory /etc/network/interfaces.d


auto lo

auto eth0


iface lo inet loopback

iface eth0 inet dhcp


allow-hotplug wlan0

auto wlan0


iface wlan0 inet dhcp

        wpa-ssid “Mom’s 6”

it turned out I was missing auto eth0 in the interface.d file.

Other useful commands: 

ifdown <config>

ifup <config>

Where config = wlan0 or eth0

Challenge – Monitoring Three-Wire Single Phased System

Three-Wire Single Phased System

I noted in a previous post,  North American houses are wired with a three-wire single phased system

Pasted Image 9 20 17 3 50 AM

Learning Resources:

Measuring Power

The OpenEnergyMonitor reference does an excellent job describing the setup of CTs for their products:

 As well as details on how to calculate power with this configuration:


The OpenEnergyMonitor reference notes: “Because there are three wires (discounting the protective earth conductor), classical theory dictates that two wattmeters are needed, and for ‘wattmeter’ read a pair of voltage and current measurements. Unfortunately, the emonTx has only one voltage input, therefore a compromise is required. Fortunately, because the voltage balance between the two legs is good, little error is introduced by assuming the voltages are equal in magnitude. However, two current measurements are always necessary.” 

The ATM90 FeatherWing measure both CT and Voltage for a circuit.  This means, besides two CTs, two voltage transformers are needed.

Testing 3-Wire Single Phase Power Measurements with ATM90 FeatherWings


I am using:

  • 2 ATM90 FeatherWings.  The Arduino Sketch talks to each through SPI.  One FeatherWing is set to CS =0 and the other to CS = 15.
  • 2 SCT-013-000 CTs  (data sheet).
  • 2 9V AC-AC Transformers.


I wrote WholeHousePowerReading.ino to read the two power measurements and send the sum of the two over mqtt.

I decided to send only the power readings:

void myEnergyMonitor_getReadings() {
  // Check to see if the ATM90 is working...if the system status returns 2, it is...
  bool atm90needsReset = false;
  if ( (eic0.GetSysStatus() != 2) || (eic1.GetSysStatus() != 2) ) {
    atm90needsReset = true;
  if (atm90needsReset) {
    strcpy_P(energyReadingJSON, msgRESET);
  int wholePower = eic0.GetActivePower() + eic1.GetActivePower();
  // Make JSON string
  strcpy(energyReadingJSON, "{\"p\":");
  char buf[10];
  itoa(wholePower, buf, 10);
  strcat(energyReadingJSON, buf);
  strcat(energyReadingJSON, "}");

I’ve gone through several iterations on what values I send to the server/cloud for visualization.  On one hand, the more info, the better.  So bring everything that is read back and store.  On the other hand, the focus is on understanding power usage.  Over time, I’d love to evolve energy monitoring to include device detection and perhaps other things that evolved understanding of a technology brings.  However, for now, I’m going to focus on whole house power.  This will be an Int sent as a mqtt message to em/# topic:

// Topic = em. Specific node = 1.  1 will map to an entry in the Address table to the house address for node 1.
const char* mqttTopic = "em/1";

The firmware sends two types of mqtt messages:

  • Info
// MQTT info messages
const char msgBOOT[] PROGMEM = "{\"m\": \"Boot\"}";
const char msgRESET[]  PROGMEM = "{\"m\": \"Reset\"}";
  • Power Readings (see myEnergyMonitor_getReadings() ).


I’ve evolved the SQLite database definition on the Raspberry Pi to three tables: 

class Reading(peewee.Model):
    power = peewee.IntegerField()

    class Meta:
        database = database

class Info(peewee.Model):
    time = peewee.TimestampField()

    class Meta:
        database = database

class Address(peewee.Model):

    class Meta:
        database = database
  • Address Table: An entry into the Address table is inserted by a person.  Here’s mine:
import peewee
from EnergyReadingModel import Address
addressID = 1
firstName = 'Margaret'
lastName = 'Johnson'
streetAddress = '555 Main Street'
city = 'Kirkland'
state = 'WA'
zipCode = '98033'

    print("Address record already exists.")
    print("Address record created.")

The addressID is part of the mqtt topic the Energy Monitor sends.  For my Energy Monitor, the mqtt topic is em/1.

  • Info Table: an entry is added to the Info Table when an Energy Monitor sends an mqtt message like:
// MQTT info messages
const char msgBOOT[] PROGMEM = "{\"m\": \"Boot\"}";
const char msgRESET[]  PROGMEM = "{\"m\": \"Reset\"}";
  • Reading Table: The power readings that are picked up from Energy Monitors.

Python App on Raspberry Pi  

The Python app on the Raspberry Pi – – picks up the energy monitor’s mqtt message and inserts the content of the payload into the right table.

def on_message(client, userdata, msg):
    # Parse out the Energy Monitor ID
    topicHead,addressID = msg.topic.split("/")
    # handle the mqtt payload
    json_data = str(msg.payload)
    data = json.loads(json_data)
    # The mqtt message could be for either the Reading or Info table.
    try: # Try treating message as an Info message
        messageValue =  data[u'm']
        timeValue = time.time()
            print("could not create a info record")
        # The message could be an energy reading or something I don't know how
        # to handle.
                powerValue = data[u'p']
                timeValue = time.time()
                print("could not create a reading record")

It Works

I can read both wires that are required in the 3-wire single phase system that is used to “pump” electricity into our house.  My next concern is the strength of the wiFi radio on the Feather Huzzah….



Until next time.  Thank you so much for reading this far.  Please find many things to smile about.

Challenge – Getting Readings of 0 from the ATM90E26

I noticed at some point the ATM90 started returning 0’s for the results.  The datasheet notes – in 4.1.3 TIMEOUT AND PROTECTION – : A read access to an invalid address returns all zero.  Hmmm…

Thanks To Those That Went Before

 Tisham Dhar quickly answered a question I had after debugging that pointed directly to what the problem was.  I want Tisham to be successful and believe he does quality work.  I highly recommend the ATM89e26 energy monitor kits.


Things I did to debug/fix:

  • Change the Energy DB model to have two tables, one for the readings, one for log information.  The log information will give a way for the firmware to let me know about events going on in the code that are worth looking at.  I added two events to the Arduino firmware: 1) Reset. 2) Boot.  I like having a way of informing me about events in the firmware.  I think sending simple mqtt messages is an easy-peezy way to do this.
  • Modified the Arduino code in EnergyMonitorStuff.h to check a reading and reset if all the values are 0.  If they are, reinitialize the ATM89.  I like having this in the code regardless.  
  if (sysStatus == 0 && voltage == 0.0 && current == 0.0 && pf == 0.0 && power == 0) {
    strcpy(energyReadingJSON,"{\"m\": \"Reset.\"}");
    return (energyReadingJSON);

it also sets up a “Reset.” mqtt message for insertion into the EnergyMonitor.db database.  The other Info I currently collect is when the firmware boots:

void setup() {

  myMQTT_publish("{\"m\": \"Boot.\"}");
void loop() {
  char *json = myEnergyMonitor_getReadings();
  delay(10000);  // take a reading every 10 seconds.


The image is rather small.  It is a plot showing power readings (measuring our microwave).  The dots represent resets (upper) and reboots (lower).  For an ~ 15 hour time period, there were 56 resets and 6 reboots.

pi@raspberrypi:~/EnergyDB $ python

Readings: 2724 Resets: 0, Reboots: 0

 Shoot…I did not take a reading prior to “fixing” the challenge so there are no resets or reboots in the infoquery results.


The cause of the 56 resets and 6 reboots was noise from the power supply.

NewImage       NewImage

Instead of using a separate power source for the FeatherWing, I was using the 9V AC-AC transformer.  No resets/reboots happened when the FeatherWing was powered through the USB port.


Ideally, only one power supply is needed to handle both the FeatherWing as well as the 9V AC-AC power measurement requirements.  I want to explore the characteristics of the DC power being provided to the FeatherWing.  Is there something in the power circuit that could be fixed in order for the 9VAC in to 3.5VDC out to be smoother?  My first step here would be to check this out using a scope.


Well, there it is.  Challenge: a lot of reboots and resets.  Originally, I thought this might be because of that darn ATM90.  Nope.  Power supply.  Of course, after the debugging this makes TONS of sense.  I have enjoyed the results of testing for myself with convincing results.

HappyDay Neighbors Web Service Side – Let’s Give Raspberry Pi a Twirl!



I’m getting my head around the collecting and visualizing of power readings.  As in, it’s not hard to collect a lot of power readings.  But what should be displayed and how should it be displayed?  This post is about coming up with a self contained, easy to use data collecting and visualization software that moves my thoughts farther in the direction of what concerns I need to think about before I figure out the best way to build an awesome experience.  I find it is first best to try stuff out.  This way I gain knowledge on what’s the best way to build what I want to deploy to others.  In my previous attempt at building a prototype back end for collecting and visualizing energy data, I used AWS Services. 


Bumbling through the AWS services to do what I want felt like I was wading through something that is gooey, muddy, and slimy.  In fact, I bumbled around so successfully that apparently in just one day I had blown through my free level and started to get charged.  Amazon’s charging capability – seemingly down to the micro transaction – is a true testimonial in their ability to suck up every penny on the table.  Here’s part of a mail I got from AWS billing when I asked about charges: Although your account is eligible for the Free Tier, your usage charges were based on activity outside the specifications of the offer. Please note that the AWS Free Tier provides up to 750 hours per month of a single-AZ t2.micro.elasticsearch or t2.small.elasticsearch instance and 10GB per month of optional Amazon EBS storage (Magnetic or General Purpose), while you were exceeding the limitation of hours.  Hmmm…..I just want to collect energy data and draw a line chart here, a bar chart there….



So I changed direction on technologies to use for collecting and visualizing energy data:

In this prototype exploration, I  use a Raspberry Pi 3 to collect readings coming in from the energy monitor using mqtt.  The Raspberry Pi will also act as a web server to display visualizations of the energy data.  I’ll use Python as the programming language and hook together:

  • mosquitto – An mqtt Broker.
  • peewee – Makes database inserts/queries easy.
  • Dash – Puts together “the best of” Flask and plotly to make web pages with data visualization.

The Goal

The goal of this post is to build what I call a “sacrificial draft” of the Collect and Visualize component of the HappyDay Neighborhood project.  I will capture mqtt messages from a Monitor reading the current used by a lamp as I turn the lamp off and on.  This is a simple prototype that will give me an idea how all the “goo” fits together.

Step1:Install the Raspbian OS

Ooh – Raspberry Pi…I feel like I get to play with the cool kids.  I haven’t played around with one.I’ll use a Raspberry Pi as the “command device.”  It will receive mqtt messages from energy monitors and store them into a database.  My browser will access a web page on the Raspberry Pi to look at the power readings.  I got a Raspberry Pi in the mail.  Next I need to install an OS on it.  


I have a natural instinct at this point to start installs at the Adafruit Learning Center.  I wasn’t disappointed.  I found and followed along.  I decided to give Raspbian Stretch Lite a twirl.  I made an image…and then…hmmm…how do I work on a headless Raspberry Pi? I don’t want to use a separate monitor and keyboard…but still the Raspberry Pi wouldn’t boot.  The challenge was the original format of the SD card.  I bought the cheapest SD micro I could find on Amazon.  I didn’t format prior to using Etcher to burning the Raspbian image.  What worked for me was first formatting the card with SDFormatter (overwrite).

Step 2: Read/Write Files to the Raspberry Pi

I am not fluent with “Unixy” copying and saving files between a Raspberry Pi and a Mac.  I wanted to use my Mac editors on files stored on the Raspberry Pi.  The way to do this is by extending the Mac file system “goo” to include the Raspberry Pi’s drive.

To be able to drag/drop/open files on the Raspberry Pi from within my Mac’s Finder, I:

    • Installed SSHFS from here.
    • Created a directory named ‘mount’ within my home directory.  I.e.: in my case this would be /Users/margaret/mount .
    • Opened a Terminal window and ran the command:
sshfs pi@raspberrypi.home: /Users/margaret/mount

Where the IP address is the unique IP address of the Raspberry Pi within my internal network.

Note: The ssh connection will break when the connection is lost – say because the Raspberry Pi or the Mac was rebooted.  So the sshfs command will be run quite often.

Now the Raspberry Pi’s storage appears as a disk within the Mac’s Finder.

Pasted Image 10 9 17 3 32 AM

pi@raspberrypi:~ $ pwd


Looks like the directory on the Raspberry Pi associated with the mount folder on my mac is /home/pi.

Step 3: Install Mosquitto

Moving right along, I need an mqtt broker on the Raspberry Pi to grab the energy monitor mqtt messages.  For this I’ll use mosquitto.  Mosquitto seems to be THE mqtt software to use on the Raspberry Pi.

Run the following on the Raspberry Pi’s command line:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get install mosquitto mosquitto-clients
pip install paho-mqtt 

Step 4: Test mqtt

OK.  Mosquitto is installed…but does it work?  I tried a few different scenarios to check this out.

Test 1 – command line

There are two command line utilities, mosquitto_pub and and mosquitto_sub.  As with all the other Linux commands I’ve used – get help on each by typing

~ $ mosquitto_pub –help

  • Open a terminal window, logon to the Raspberry PI and type the command:

 $ mosquitto_sub -t test/test1  -d

I.e.: subscribe to mqtt messages brokered on localhost sent to the test/test1 topic.  Debugging info (-d):

Client mosqsub/17589-raspberry sending CONNECT

Client mosqsub/17589-raspberry received CONNACK

Client mosqsub/17589-raspberry sending SUBSCRIBE (Mid: 1, Topic: test/test1, QoS: 0)

Client mosqsub/17589-raspberry received SUBACK

Subscribed (mid: 1): 0

  • Open a second terminal window, logon to the Raspberry Pi and type the command:

$ mosquitto_pub -t test/test1 -m “test message” -d

Client mosqpub/17627-raspberry sending CONNECT

Client mosqpub/17627-raspberry received CONNACK

Client mosqpub/17627-raspberry sending PUBLISH (d0, q0, r0, m1, ‘test/test1’, … (12 bytes))

Client mosqpub/17627-raspberry sending DISCONNECT

The terminal that was running mosquitto_sub received an mqtt message:

Client mosqsub/17589-raspberry received PUBLISH (d0, q0, r0, m0, ‘test/test1’, … (12 bytes))


test message

Test 2 – python


As an aside, I just checked /usr/bin on my Raspberry Pi and did ls -l python* :

lrwxrwxrwx 1 root root 9 Jan 24 2017 /usr/bin/python -> python2.7
lrwxrwxrwx 1 root root 9 Jan 24 2017 /usr/bin/python2 -> python2.7
-rwxr-xr-x 1 root root 3162224 Jan 19 2017 /usr/bin/python2.7
lrwxrwxrwx 1 root root 36 Jan 19 2017 /usr/bin/python2.7-config -> arm-linux-gnueabihf-python2.7-config
lrwxrwxrwx 1 root root 16 Jan 24 2017 /usr/bin/python2-config -> python2.7-config
lrwxrwxrwx 1 root root 9 Jan 20 2017 /usr/bin/python3 -> python3.5
-rwxr-xr-x 2 root root 3976264 Jan 19 2017 /usr/bin/python3.5
-rwxr-xr-x 2 root root 3976264 Jan 19 2017 /usr/bin/python3.5m
lrwxrwxrwx 1 root root 10 Jan 20 2017 /usr/bin/python3m -> python3.5m
lrwxrwxrwx 1 root root 16 Jan 24 2017 /usr/bin/python-config -> python2.7-config

Jeepers! I had no idea so many versions of Python were installed. For now, I defaulted to using python2.7


I tested out running mosquitto commands in python using the script on this page.  Here are some of the results I was getting:

$SYS/broker/connection/Usuario-PC.bridgeConnection/state 1

$SYS/broker/connection/Lucas-PC.bridgeConnection/state 1

$SYS/broker/connection/Servidor-PC.bridgeConnection/state 1

$SYS/broker/connection/caixa02.bridgeConnection/state 1

after awhile I got this error:

  File “”, line 25, in <module>




  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 1481, in loop_forever


    rc = self.loop(timeout, max_packets)


  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 1003, in loop


    rc = self.loop_read(max_packets)


  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 1284, in loop_read


    rc = self._packet_read()


  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 1849, in _packet_read


    rc = self._packet_handle()


  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 2305, in _packet_handle


    return self._handle_publish()


  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 2500, in _handle_publish




  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 2647, in _handle_on_message


    self.on_message(self, self._userdata, message)


  File “”, line 13, in on_message


    print(msg.topic+” “+str(msg.payload))


  File “/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/”, line 360, in topic


    return self._topic.decode(‘utf-8’)


  File “/usr/lib/python2.7/encodings/”, line 16, in decode


    return codecs.utf_8_decode(input, errors, True)


UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0xe7 in position 31: invalid continuation byte

I’m thinking there are some characters in the topic name that can’t be parsed.  For now I’ll ignore and consider the test successful.

Test 3: Mac to Raspberry Pi Test

Can I publish an mqtt message on my Mac and have the Raspberry Pi get it? Easy-Peasy!

import paho.mqtt.publish as publish
# Based on this example:
publish.single(“test/topic”, “Hello Pi”, hostname=”″)

Whoa.  That was too easy!

Step 5: Store Incoming Energy Messages

Store mqtt messages into a database on the Raspberry Pi.  FINALLY some fun, yah?  According to this post which I’ll take as fact until I learn better: SQLite is built into Python but is only built for access by a single connection at a time.  For now, I’ll use the built in SQLite.  When I scale to include other houses, I’ll have to change to a different database.  I should have a better feel with what data to store, how to store it, and what visualizations work the best.

The other piece of software I’ll  use for database access is the peewee ORM.  You might want to take a read at the material at this link –  if you are new to using the peewee ORM.  I chose to use the peewee ORM because of it’s simplicity.  

Install peewee

$ pip install peewee

Create the EnergyData database

For this step, I wrote  It contains the definition of the power and time variables and creates the EnergyReading.db file. I used Atom as my editor and saved the file to the EnergyDB directory I created (within the mounted directory).

~/EnergyDB $ python

Created the EnergyReading.db file. This will be the database where the Energy readings will be stored.

Send Power Readings

The Feather Huzzah with the ATM90 FeatherWing is running the SendEnergyReadingsUsingMQTT.ino Arduino code.  For this part, I needed an mqtt library for the Arduino.  I am using what seems to be the most popular –  pubsubclient.

SendEnergyReadingsUsingMQTT.ino – 

  • Reads the power, current, power factor, and voltage.
  • Sends the energy Readings as an mqtt message formatted in JSON.  I ended up using a very simple name/value format.
    • {“sysStatus”:2,”voltage”:124.5,”current”:10.5,”power”:1278,”pf”:0.75}
    •  I used an online JSON parser to understand how JSON strings are parsed…

I use a 100 byte buffer to store the message.  This is lazy of me – what if the message is longer than 100 bytes?  I decided on 100 bytes after calculating the number of bytes in the above JSON string using this letter counting calculator:


Since this sample was 67 characters, I assume the JSON string will never be longer than 100 bytes.

Add Power Readings to the Energy DB 

I run the python app on the Raspberry Pi. subscribes to the EnergyReading/# topic.  When it receives a message, it puts it into the energy db.

Note: while writing the python app, I learned about timestamps…which made sense to me to use as the time the energy reading was created.  I did it this way because the Raspberry Pi syncs its clock with ntp.  This means I can use the clock on the Raspberry Pi and not add an additional complexity of a RTC to the Energy monitor. 

>>> import time

>>> ts = time.time()

>>> print(ts)


>>> import datetime

>>> st = datetime.datetime.fromtimestamp(ts).strftime(‘%Y-%m-%d %H:%M:%S’)

Run As a Service

I want to capture readings on the Raspberry Pi 7 x 24.  In order to do this, I need to set up the Python script to run as a background service.  I found Matt’s post – How To Autorun A Python Script On Boot Using systemd to be very helpful.

Here’s the content of the systemd (addMessageToEnergyDB.service) file I’m using:


Description=Add Energy Record to EnergyDB.





ExecStart=/usr/bin/python /home/pi/EnergyDB/



Executing the systemctl status command on the service gives me an indication readings are being received and entered into the EnergyDB:

 $ sudo systemctl status  addMessageToEnergyDB.service

addMessageToEnergyDB.service – Write Energy Record to the CSV file.

   Loaded: loaded (/lib/systemd/system/addMessageToEnergyDB.service; enabled; vendor preset: enabled)

   Active: active (running) since Sun 2017-11-12 09:20:01 PST; 9min ago

 Main PID: 634 (python)

   CGroup: /system.slice/addMessageToEnergyDB.service

           └─634 /usr/bin/python /home/pi/EnergyDB/


Nov 12 09:26:15 raspberrypi python[634]: (‘INSERT INTO “reading” (“addressID”, “v1”, “i1”, “p1”, “pf1”, “v2”, “i2”, “p2”, “pf2”, “time”) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)’, [1, 123.92, 2.508, 284

Nov 12 09:26:36 raspberrypi python[634]: (‘INSERT INTO “reading” (“addressID”, “v1”, “i1”, “p1”, “pf1”, “v2”, “i2”, “p2”, “pf2”, “time”) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)’, [1, 123.89, 2.593, 294

Step 6: Make an HTML5 Web Page

I originally started down a path of Flask + plotly.

Then I figured out Dash was a better choice for visualizing energy readings on a web page.  I found this video gave me a good feel why I decided on Dash.  Dash builds on top of Flask and plotly so I get all the great features of these frameworks with the added benefit of doing everything in Python.  I  installed Dash using the directions in Getting Started.  Here is an image of the plot of energy readings I got after taking energy readings for our microwave over time:

I wrote and ran the python code.  Whoa.  Pretty darn simple.  At least getting started is not hard.  I’ll find out how easy it is to do more complex visualizations soon enough! 🙂
if __name__ == __main__:

Since I’m running this on the Raspberry Pi, I set the host to instead of the default (localhost). 

That’s it for now 

My-oh-my it took me a bit of bumbling to get to this point.  The learning part was thrilling.  I’m sure I’ll have many changes to my plans as I continue.  Next, I want to run the python script that adds mqtt messages to the Energy DB as a background service that starts up on boot.  Also, for some reason after “some amount” of time, the Feather starts getting 0 for all the energy readings…hmmm…. I never expect stuff to “just work.” 



Thanks for reading this far.  Please find many things to smile about.




Trying out the ATM90E26 FeatherWing

I’m excited! I just received Whatnick’s ATM90E26 FeatherWing that I ordered on Tindie.  I got this while I wait for the ones I ordered from Crowd Supply.  I guess I got a bit order happy.  I wanted to support Tisham’s work.

By the Way – I ran across Tisham’s discussion of his work at the Adalaide Maker Faire.  It helped give me perspective.

Spoiler alert – It works great.

The Goal

The goal of this post is to run the Arduino sketch provided by WhatNick on a Feather Huzzah with the ATM90E26 FeatherWing and get expected results for the readings.  I will use the UART option to communicate between the Huzzah and the ATM90 based on this comment made on WhatNick’s Tindie pageThe ATM90E26 has the flexibility to be accessed both over SPI and UART. However the SPI mode it supports is only Mode 3, which is an unsupported mode of the ESP8266 Arduino stack. So I ported my ATM90E26 Arduino interface code over to UART mode with CRC check and everything after a few days of head scratching.

The Arduino sketch I will run is ATM90E26_UART.ino found at this GitHub location.  

Thanks To Those That Went Before

  • To Whatnick for building the ATM90 FeatherWing and writing a very simple to use library for reading/writing to it’s registers using the Arduino IDE.  So far I am very impressed with the quality of work.  From the choice of components, PCB material, soldering, and design.  What a delight.  A HUGE THANK YOU to Tisham.  Without Tisham’s help, I would not have been successful.  THANK YOU.  Oh – and THANK YOU.
  • To Home Depot for carrying just what I need for my DIY projects (except the stuff I need from Adafruit…By the Way – can the two of you merge?) – ESPECIALLY FOR KEN. Ken used to head an electronics shop with 1,000’s of employees.  He works in the electronics section of Home Depot several mornings.  Ken is EXTREMELY knowledgable, helpful, and has stories/no stuff that is just interesting to listen to.  So much so that Home Depot is my favorite place to go. 


Test Tools

  • Kill A Watt – for comparison current and voltage measurements.  I’ve used this many times.
  • Current Clamp – another tool to compare readings.  This works in the same way the CT works.  

In order to test the ATM90 with the clamp, I made a line splitter.  With this little beauty of a tool, I can check the current use of any device.  I don’t have to go to the breaker box and muck about and mess up the electricity in a way that bothers other family members.



Making the line splitter was fun/simple.   

  • 1’ green, white, black 14AWG wire $.23 * 3 = $.69
  • 3 wire female plug $2.79
  • 3 wire male plug $3.29
Total cost: $6.77.
Just follow the color coding.  Reminds me of coloring pictures that contained info on what color I should use (and by the way – I did not appreciate being told what colors to choose.  I would choose what I wanted.  Did a picture of the sky HAVE to be blue?).
Coloring info from this web site:

Adjusting the Gain

Before I can start accurately measuring voltage and current, I need to calibrate by setting the UGain and IgainL.  This is discussed in section (p. 14) of Atmel’s M90E26 app note.  Since I don’t know what I am doing, I asked Tisham about this.  His reply:

  • There are some videos that may be helpful (yes they are).  If I remember correctly, the first video was great at showing what values Tisham got when running test code.  He also mentions Ugain and Igain.
  • Do not use the calculation for Ugain that is in the app note.  Rather: new_Ugain = existing_Ugain * (expected_Voltage/current_Voltage)
  • calculate new_gain, update register, then make sure to update checksum 2.  If checksum 2 is not updated, the results are bad.
  • After the voltage is calibrated you can use a purely resistive load with known power usage or another meter to calibrate the current gain. The value already in the script should be pretty close.  
    • new_Igain = existing_Igain * (expected_power / current_Power)

Calculate Ugain

new_Ugain = existing_Ugain * (expected_Voltage/current_Voltage)

  • existing_Ugain is set in energyic_UART.cpp CommEnergyIC(0,Ugain,0xD464); //Voltage rms gain
  • expected_Voltage was measured at 124.5V
  • current_Voltage = 192.93.  I got this value from running ATM90E26_UART.ino:
Sys Status:2
Meter Status:801
Active power:11.00

new_Ugain = 0xD464 * (124.5/192.93) = 54372 * (124.5/192.93) = 35086.891618722 = 35087 = 0x890F

Update Checksum 2

After changing the Ugain setting in energyic_UART.cpp to 0x890F, I reran the Arduino sketch.  Checksum 2 will be off, but as the data sheet points out: The readout value of the CS2 register is the calculation by the M90E26…

The library already writes out the values for the checksums and lets me know if there is an error.  There is indeed an error for Checksum 2.  The new value for Checksum 2 that I read: 0x428C.  I modified this line in energyic_UART.cpp from 

CommEnergyIC(0,CSTwo,0xD294); //Write CSTwo, as self calculated

CommEnergyIC(0,CSTwo,0x428C); //Write CSTwo, as self calculated 

Check Voltage Reading

Here are the results I got after running the Arduino sketch;

Sys Status:2
Meter Status:801
Active power:7.00

The voltage I measured was 124.5V.  The voltage I’m getting back from the ATM90 is 123.6V.  My thoughts:

  • The reading I am getting back from the ATM90 is “close enough” to the reading I get from the Kill-A-Watt and multimeter.
  • The calculations will not be precise.
  • At this point, I’m not spending any time evolving my knowledge of the amount of error.


I compared current readings from several devices.  The current measurements – for the ATM90 compared with the clamp were the same, if not extremely close.  For this reason, I am not adjusting the IgainL.



ChallengesBringOpportunites YIPPEE………..IT WORKS!!!!!


WhyCat hmmm… this works for one cable…but I have two:

IMG 6215


Prototyping the HappyDay Neighbors Energy Monitoring Project


I’m thinking through the client side of my neighborhood energy monitoring project.  I’m starting to call this project HappyDay Neighbors.  Don’t know if that will be the final name…Naming is hard.  

The energy monitoring experience includes collecting energy readings from our breaker boxes, sending the readings using mqtt to a data store in DAH CLOUD.  The home owner then views the readings.  Through viewing the readings, the home owner will understand how much energy is being consumed and think of ways to decrease energy use.  I also want to explore social engagement using gaming/facebook/twitter…The social part is TBD.  My current focus is on reading and visualizing the data.

The Goal

The goal of this post is to explore a path to visualizing the energy readings.  I’ll do this by building a prototype that:

  • grabs 8,000 energy readings from Tisham’s data on Thingspeak and sends it to a cloud data store using mqtt.
  • shows a histogram of the energy readings in a web browser.

Thanks To Those That Went Before

  • The folks behind the abundant resources and excellent community of the OpenEnergyMonitor project.  The folks in the community as well as the resources are very accessible.  There are many of us who benefit greatly from their ongoing efforts.  I’m excited!  Just today I got an emonPi in the mail.  Hopefully the CTs I have on order will arrive soon.  Then I’ll be able to give the OEM experience a twirl.
  • A HUGE shout out to Tisham Dhar.  Tisham is the creator of the ATM90E26 Single-Phase Energy Monitor Dev Kits.   I have been impressed with the intelligence and care Tisham has taken to Open Source his work.  Tisham also kindly answered all the questions I peppered him with.  His work looks super sharp to me.  

Background Info

My exploration took me on a path that relied on AWS services.  I went down this path because:

  • eventually I want to scale collecting and analyzing energy readings throughout our neighborhood and beyond.
  • I wanted to build as quick and dirty as possible.  This meant relying on a system of services – mqtt broker, data base, graphing – that were loosely coupled together, had a lot of folks using it so that there were a lot of videos and docs on use cases.
  • I am using the prototype to better define the final experience.
When I started this post, I had no idea what AWS services to use.  My learning process was to jump right on in.  I’ll write a bit about what I’ve learned at the end of this post.

There was a lot of goo…um…technology…that I was unfamiliar with. I bumbled my way through relying on mostly these resources:

Open Source

These files are used in the post:

Part 1: Sending mqtt Messages

I wanted to divide and conquer taking energy readings using sensors/hardware from visualization.  This way, I can start  collecting energy readings prior to having the hardware reading.  It will also allow me to focus on the why’s and how’s of the data visualization step.  I’ll read power readings from Tisham’s Thingspeak feed. I started discussing Tisham’s excellent work in this post.

Step 1: Build a File of Power Readings

I did this in my first Jupyter notebook.  As I started putting together the workflow of this prototype, it became apparent that date conversions are hard to get right (…yah, no kidding 🙂 ).  The updated notebook – ReadTishamDataIntoCSV.ipynb.

Sending date/time strings are tricky. Computing loves numbers, humans love strings.  I convert the date/time string to it’s epoch value (a long datatype).

The format TIsham uses to store the date/time format:


Sending date/time strings are tricky. Computing loves numbers, humans love strings.  I convert the date/time string to it’s epoch value (a long datatype).  Here’s the Python script I ended up using to take in the dates in the ThingSpeak feed and convert to an epoch format:

dt = datetime.datetime.strptime(entry[u’created_at’],’%Y-%m-%dT%H:%M:%SZ’)
epoch = datetime.datetime(1970, 1, 1)
epochTime = int((dt – epoch).total_seconds())


Entries in the CSV file look like this:


i.e.: Energy reading in watts, epoch time when measurement was taken.

E.g.: converting an epoch time format to a human readable format using Python:

>>> import time

>>> time.strftime(‘%Y-%m-%d %H:%M:%S’, time.localtime(1506532776))

‘2017-09-27 10:19:36’

Step 2: Send the Power Readings as mqtt messages 

In order to send mqtt messages, I needed to figure out what mqtt broker to use.  My requirements for a broker included robust availability, storage and visualization of the energy messages.  I ended up choosing AWS services.

I put the code I used into the Jupyter notebook SendMqttToTishamDataTopic.ipynb (located at this GitHub location).  It sends the mqtt energymqtt messages to the topic /clientPrototype/data/Tisham.

To do this I needed boto3: and AWS_CLI.

  • Install the AWS CLI:  From the web page – The AWS CLI is an open source tool built on top of the AWS SDK for Python (Boto) that provides commands for interacting with AWS services. hmmm…I have the Anaconda variant of Python 3.6 installed.  So this seems to mean I’ll be need to use the conda package manager (instead of following Amazon’s instructions, I followed instructions from stack overflow):

$ conda install -c conda-forge awscli

$ conda install -c anaconda boto3

I needed an access key and secret access key.  I followed the direction to get the access key and secret access key from this Quick Configuration Link.  Next I ran aws configure and copy / pasted the keys.  Here’s what the mqtt messages look like:
{'power': '176', 'time': '1506532776'}
{'power': '177', 'time': '1506532795'}

Part 2: Data Store and Visualization

For data store and visualization I used:

  • the AWS Elastic Search Service as the message store.
  • an AWS IoT rule that triggered when an mqtt message with the /clientPrototype/data/# topic come into the broker.
  • AWS Kibana for visualization.
It is not my goal to give tutorials on the AWS services I used.  While overwhelming at times, once I focused on each one I was able to bumble through using the available videos and tutorials.  I’ll just go through the steps I used after many starts and stops.

AWS Elastic Search Service

  • I created a test domain.
  • If the clientPrototype index (database) exists (from prior testing) delete:
  • Create the clientprototype index and powerReading type:
curl -i -X PUT
  "mappings": {
    “powerReading": {
      "properties": {
        "time": {
          "type": “date",
          “format": “epoch_second"
        “power": {
          "type": “long"
' ''


I followed the example in the tutorial:

  • As I mentioned above, I’m using the epoch date format.   Stuff that took me time to figure out:
    • Removing line breaks from the curl command:
      • Copy/pasted the curl command to create the powerReadingtype within the clientPrototype index of the test domain into TextWrangler to edit.  
      • Remove line breaks using the Remove Line Break option on TextWrangler under the Text menu.  
    • Figuring out the epoch date formatting – both on the Python side as well as the format to use in Kibana. 

 AWS IoT Rule

Here’s the rule I created in the AWS IoT Rule UI:



when a message comes into the /clientPrototype/data/<anything> topic, send it on to Elastic Search.  The topic I set up in the SendMqttToTishamDataTopic.ipynb notebook, /clientPrototype/data/Tisham , will trigger this rule when the mqtt message comes into the AWS broker.  


The action taken:

Pasted Image 9 29 17 10 39 AM


Rules run in the system using a role set up within the IAM UI. 


I gave the following permissions:


Access to the CloudWatch logs is important for debugging.  I’ll get to that.  The other policy allows the rule to publish into ElectricSearch:



Sending mqtt messages and publishing into ElasticSearch can now be testing.


The test will:

  • use the AWS IoT test UI to see if the AWS broker is receiving the mqtt messages.
  • evaluate logs in the log file to verify power readings are published into Elastic Search.

AWS IoT Test


Evaluate Logs

The AWS services – IoT (message broker) and ElasticSearch can be configured to log debug information.  Monitoring is done through the AWS CloudWatch service.

I couldn’t get it to work until I went through the AWS_CLI command:

aws iot set-logging-options –logging-options-payload roleArn=”arn:aws:iam::XXXXXXXXXXXX:role/service-role/pub_to_es”,logLevel=”DEBUG”

Here are the log entries that are created when the mqtt message is sent:

2017-09-30 17:17:13.659 TRACEID:f872acf1-ccaf-b6f0-6df4-f94a7e91c3c5 PRINCIPALID:XXXXXXXXXX/XXXXXXXXX/AIDAIHBTGHRY27IMGPW2M [INFO] EVENT:PublishEvent TOPICNAME:/clientPrototype/data/Tisham MESSAGE:PublishIn Status: SUCCESS

2017-09-30 17:17:13.659 TRACEID:f872acf1-ccaf-b6f0-6df4-f94a7e91c3c5 PRINCIPALID:XXXXXXXXXX/XXXXXXXXX/AIDAIHBTGHRY27IMGPW2M [INFO] EVENT:PublishEvent MESSAGE: IpAddress: SourcePort: 51507

2017-09-30 17:17:13.712 TRACEID:f872acf1-ccaf-b6f0-6df4-f94a7e91c3c5 PRINCIPALID:393111621300/AIDAIHBTGHRY27IMGPW2M/AIDAIHBTGHRY27IMGPW2M [INFO] EVENT:MatchingRuleFound TOPICNAME:/clientPrototype/data/Tisham CLIENTID:N/A MESSAGE:Matching rule found: ClientPrototypeRule

2017-09-30 17:17:13.712 TRACEID:f872acf1-ccaf-b6f0-6df4-f94a7e91c3c5 PRINCIPALID:393111621300/AIDAIHBTGHRY27IMGPW2M/AIDAIHBTGHRY27IMGPW2M [DEBUG] EVENT:ElasticsearchActionStart TOPICNAME:/clientPrototype/data/Tisham CLIENTID:N/A MESSAGE:Starting execution of ElasticsearchAction on topic /clientPrototype/data/Tisham

2017-09-30 17:17:13.843 TRACEID:f872acf1-ccaf-b6f0-6df4-f94a7e91c3c5 PRINCIPALID:393111621300/AIDAIHBTGHRY27IMGPW2M/AIDAIHBTGHRY27IMGPW2M [INFO] EVENT:ElasticsearchActionSuccess TOPICNAME:/clientPrototype/data/Tisham CLIENTID:N/A MESSAGE:Successfully indexed document in ES. Message arrived on: /clientPrototype/data/Tisham, Action: elasticsearch, Endpoint:, Index: clientprototype, type: powerReading, id: 008edff0-11f1-46c3-9e84-76500a5a4771

The log entries are a HUGE help during debugging.  It took me many (many) attempts to get through the workflow.  The log files made it obvious what the problem was.  The problems typically were 1) ill-formed JSON 2) lack of security clearance.

Kibana Visualization

A link to Kibana is available in ElasticSearch once I created the test domain:

Pasted Image 9 29 17 1 02 PM

Once I wrapped my head around Kibana’s main use case – visualizing a gazillion data millions upon millions (e.g.: all the flight data from the past few years…).  I was able to wrap my head around making this bar chart:


For the Y-Axis, I ended up using the MAX Aggregation figuring when the data is zoomed in or out this becomes the most interesting for a bar chart given the constraints on setting the Y-Axis.  I.e.: unlike Excel – data stores and visualizers that …well um…excel with data sets in the 10,000’s but are not focused on BIG BIG DATA – that plot individual points, Kibana uses aggregates (hence one of the suggested videos is about Kibana aggregation).

Lessons Learned 

Whew.  Well, it works.  I learned:

  • AWS services are both powerful and overwhelming.  It gets confusing what is free and what ends up being charged. For example, one path I took ended up costing about $14.  This alarmed me because I had no idea I was getting charged…things “just worked.”  I need to get a better feel of free versus costs.
  • Using AWS services is an easy way to scale to many, many energy monitors.
  • ElasticSearch and Kibana are optimized for finding a needle in a haystack. In this case, the haystack is gazillions of data points.  The needle being a behavior that is in the data but needs to be searched out.  Their scenario is one of BIG DATA.  While this is a scenario I am interested in eventually, the primary interest is for evaluating energy readings within a fairly small time frame – a day, a week, a month and/or real time.  Other AWS services for the data store (e.g.: S3) might be a better solution.  This is dependent on the eventual scenarios.  Kibana is not the best visualization because it assumes the scenario is one of a gazillion data points.  Kibana’s context is one of putting data points into buckets to display.  
  • AWS IoT rules are awesome.  It is so easy to trigger an AWS service (like storage of the payload) and set up roles.
  • mqtt works well for sending energy readings from the energy monitor that will be located in the breaker box.
Thanks for reading this far…..    Please find many things to smile about.



For the prototype – where my goal is to tweak out all aspects – hardware, mqtt message passing, storage, visualization, scaling to our neighborhood (and beyond) – I feel the path I took works out.  There are many, many choices from different vendors…and many choices within the AWS family to get to the finish line.  For now, I’m happy with this workflow.



The Wiring in our House

I continue to learn how our house is wired.  Here’s where all the breaker switches are located:

IMG 6214

We started out with the original box.  Then when we added a generator, the other panel was added.  I have been adding energy monitoring hardware goo to wires in the newer panel.  For example, I live to tell about my installation of a Sense monitor.

IMG 6215

I bought and installed the Sense because I wanted to experience a consumer ready experience and the idea of the Sense figuring out what appliances were in my house intrigued me.  How good was it at doing this?  What I enjoyed most was how easy it was for me to start tracking our energy use.  I didn’t have to figgle around with config files, go through any complex setups – it just worked.  Now I figure I can DIY the hardware and software for my HappyDay Neighbors project.

When I started exploring what CTs to use, I noted:  I measured the wire gauge of the two wires where the Sense CTs are located.  They are  1/0 AWG.  This surprised me since most cable examples I had seen on the Internet used the wider 4/0 cable.

Hold on.  Wait a minute.  It became clear to me I didn’t understand our wiring.  In particular, where the heck is the mains breaker switch?  How is the electricity really flowing in from the transformer?  What Amps? 100A? 200A?..I dunno…

Bumbling back to the original breaker box, here are the innards:

IMG 6225

Hmmm….couple of things I am not sure about:

  • I expected to see a neutral wire in the center.
  • I expected to find a mains breaker.

But I didn’t.  Is this because of the installation of the generator and the newer panel?  Where is the mains breaker?  Why isn’t there a neutral wire?

Getting the Hardware Together – the Current Transformers

I’m (still?) getting up to speed on how electricity flows through our house and how use is commonly measured. ..

The Goal

The goal of this post is to:

  • explain why I chose to buy two SCT-013-000 CTs.
  • write down my understanding of how 240V electricity comes into our house.

Our House

All the energy monitors I know about use Current Transformers (CTs) to read the voltage and current used by appliances within a house.

Our house is wired with a three-wire single phased system.  As the Wikipedia article notes: This three-wire single phase system is common in North America for residential and light commercial applications. In this case, the voltages are 120 V line to neutral (grounded center tap of transformer) and 240 V line to line. This system allows 120 volts to be supplied for lighting and convenience outlets, and 240 volts for higher-demand appliances such as space heating, air conditioning, kitchen stoves or water heaters.

I copied this image from Home Depot’s Basics of Your Home’s Electrical System video:

Pasted Image 9 20 17 3 50 AM

The Learn section of OpenEnergyMonitor has a great article on North American Power Supply characteristics.  It includes this diagram:

As is true with my house: Standard practice among US electricians is to connect the circuits to the breakers in numerical sequence, with odd numbered breakers connected to one leg and even numbered breakers connected to the other leg. Thus, half of them will be on one leg, half on the other leg, with the aim being to balance the load evenly.

The implications of North American Power Supply characteristics to the firmware are briefly covered on the Use In North America/OpenEnergyMonitor’s setup guide.

Ordering CTs

How Many

Given the split of the 240V into two 120V lines, I will need two CTs, one for each line.

What Size

I used my caliper and measured the diameter of each of the 120V cables to be .32” or 8mm.

IMG 6222

According to the AWG wire sizes, this makes the cables 1/0 AWG.  This surprised me since I most cable examples I had seen on the Internet that had similar wiring to our house was 4/0, which is significantly wider.  Perhaps this is because the panel is actually a sub-panel that was put in place when we had a generator installed.  I didn’t open the original (main) panel.  Perhaps the cables for the 2 120V lines are 4/0 in the original panel.

IMG 6214

YIPPEE! This means I can use the common SCT-013-000 (data sheet).  YHDC sells several different CTs.  The SCT-013-000 is the one sold by OpenEnergyMonitor.  I would have ordered from their store.  Unfortunately, I would have to pay to have it delivered from the UK.  Instead I ordered two from  


…so much for that…

Jupyter Notebooks are Awesome

I made my first Jupyter Notebook.  I had installed Jupyter awhile back after listening to Adafruit’s Python Playgrounds LIVE! Youtube video.  The video got me totally inspired by the Python community and the usefulness of Jupyter Notebooks.  The blog post documents my first experience creating a Jupyter Notebook.

Thanks To Those Before

  • A HUGE shout out to Tisham Dhar.  Tisham is the creator of the ATM90E26 Single-Phase Energy Monitor Dev Kits.  I just found out about Tisham and his work.  I have been impressed with the intelligence and care Tisham has taken to Open Source his work.  Tisham also kindly answered all the questions I peppered him with.  His work looks super sharp to me.  I have already learned a lot from what he has Open Sourced.
  • Adafruit for their Python Playgrounds Youtube video.  I learn quite a bit from Adafruit’s learning sections and videos.  This inspirational video got me yearning to play with Python and Jupyter Notebooks.

On To The Show

A few days back, I found out about Tisham’s fantastic work. In particular, the ATM90E26 Single-Phase Energy Monitor Dev Kits.   YIPPEE!!!…but..bummer…I won’t receive them until December 5th.

So I emailed Tisham if there was a way for me to get my hands on some of the data output (power readings over time) so that I can start working on the end user side of the Neighborhood Energy Monitoring Project I decided to start up.  Tisham kindly gave me access to his Jupyter Notebook that pulls data from his feed. 

Not letting the fact that I have an advanced beginner knowledge of Python and no knowledge of how Jupyter Notebooks work, I am happy to say after a few hours, I was able to create a Jupyter Notebook that fetched energy data from Tisham’s ThingSpeak feed, plotted the data, and wrote the data to a CSV file.


ChallengesBringOpportunites Talk about exhilarating.  I can see why I am hearing more about Jupyter Notebooks.  What a fun environment.  It reminds me of the times I used Swift Playgrounds when I was programming on the iOS.  A great thing about Jupyter Notebooks are the incredible amount of extremely powerful libraries available – for AI, machine learning, image recognition, plotting…tons of stuff I don’t understand….


Now that I have sample energy data to simulate what would be coming from DAH cloud to the web browser and/or smart phones, I can prototype this software as I wait for Tisham’s dev kits to arrive.

New Project: Neighborhood Energy Monitoring


, , ,

I decided for my next project to build an energy monitor experience for our neighborhood.  My goal is to change our behavior from wasting to saving energy.  

Hardware will be placed in our breaker boxes that monitor the voltage and current being provided to the house.  The hardware includes:

  • CTs on the mains lines (the white circle with clamps on the breakout box image)
  • Sensors, micro controller, and wifi to read in, digitize the current and the voltage, and send the readings to dah Cloud.
  • A web browser or smart phone app knows how to access the current and voltage readings that are stored in dah Cloud.  The readings are displayed as a graph.  This way, we can tell not only how much electricity is being used, but also what appliances are using the most electricity.  Using this information, we can adjust our electricity usage to bring the amount we use down.
  • A social app (on Facebook?  Twitter?  Text messages?) will reward members of our neighborhood with knowledge of how much CO2 they have saved from being released into the atmosphere – as well as other social game techniques.  Social gaming will be a method to get a social bond around energy conservation.  Perhaps from this we can decide as a group to install solar panels on our houses, etc.
I plan to use these blog posts as a way to document my progress.