Tags

, , , ,

This post builds on the content of previous posts as well as what I have picked up from the exceptionally helpful folks that have left comments, replied to questions in a variety of forums, and other helpful folks that have published their findings in their blogs and websites.

Time to Log Data

One of the features of the Base System will be to capture hydroponics sensor readings to a local file so that I can then do whatever post analysis and environmental improvements based on readings that are affecting the plants.  In this post I’ll send data from a Hydroponics Sensor Node to the Base System. The Base System will store the data on an SD card in CSV (Comma Separated Value) format.  

CSVDataFromSensors

I will then open the CSV file in Excel or Google Docs to see if I can notice any trends in the readings or fixes I should make in the pH, amount or kind of nutrients, temperature conditions, and possibly the lighting conditions.

Steps

The steps to do this include:

  •  Sending sensor readings to the Base System.  This was done in an earlier post.  While I look into some SPI misbehaviors, I’m going to use a 433 MHz TX and 433 MHz RX  to talk between the Base System and the hydroponics sensor node.  I’ll start by setting up the sketches using two USB ports on my Mac.  This makes it easier to debug because I can open up two terminal sessions and view the Serial.print() statements.  then move the sensor node to the hydroponics system.
  • Writing data to the SD card in CSV format.
  • Importing CSV file into an Excel spreadsheet.

Parts

Sketches and Code

You can download the sketches and code used in this post here.   I’ve decided against showing code in a post.  I’d rather have the code be documented enough so that you understand it.   Please let me know if you find stuff you don’t understand.  I’d like to then update the code as needed so that it is easier to use.  Of course, please suggest improvements!  I hate to remind – but my college degrees are in Geology and I spent most of my working life in management.  We can all improve with your help.

Can You Hear Me Now?

This post is focused on getting sensor readings into an analytics tool such as Google Docs and Excel.  I used some inexpensive 433MHz TX/RX components that I had lying around to send and receive the sensor data.  This YouTube video is a great way to jump start using the 433MHz chips if you haven’t already done so.  The sketches use the VirtualWire library.  You might also be interested in the active forum.  The VirtualWire library documentation notes in section 5, performance,: At 2000bps, Range over 150m  (492 ft) [mileage will vary].  My mileage definitely varied.  I saw a much smaller range.  This is most likely due to RF interference in my house.

Back to the 433MHz. I’ll be swapping these out for my “final” home sensor/automation network solution when I choose it.  I hope not to be spending too much additional time on RF exploration.  I’ve learned to a level where I am by no means an expert, but do feel comfortable with how the different RFs work at the level needed to use for a home sensor/automation network.

Goldilocks and the Three RF choices

One of the areas I wished to explore was the varying RFs to make up the home sensor network.  My goals in this exploration include:

  • gaining an understanding and familiarity at the sw/hw level of low cost home sensor/automation network RFs
  • choosing a solution for my hydroponics home sensor/automation system
The features I am looking for in a solution include:
  • low cost
  • “adequate” range (the ability to transmit from anywhere in my house and receive from the point farthest away)
  • robust Arduino software library that has the “right” APIs (“right” is determined by need and seems to vary the more I learn!)
  • low power
  • enthusiastic, helpful internet community
  • includes features such as the ability to determine signal strength, node differentiation such that multiple nodes can transmit sensor data to a Base System and the Base System will know which node sent it, and reception by the Base System of uncorrupted packets within a reasonable timeframe.  There are always going to be dropped or corrupted packets.  How easy is it for the Base System to receive uncorrupted sensor readings? 
  • requires minimal soldering on my part 

Up until now I have been exploring the capabilities of the various RF solutions, the Arduino libraries that come with them, and what level of support is available.  I’ve stayed away from solutions like zigbee and Z-Wave because of cost (true in both cases) or barriers to entry (true in the case of Z-Wave).  For example, I sent a question to the Z-Wave alliance about exploring Z-Wave.  The chairman (Mark Walters) sent the following email:

Getting involved with native Z-Wave development isn’t for the faint of heart.  To develop one must first purchase a developer’s kit from Digikey and in doing so sign a licensee agreement with Sigma Designs the IP holder of the Z-Wave technology.  The least expensive developers kit is $1,500 and then you will need a $3,500 compiler and about $1,500 worth of developers hardware.  I’m not sure if you want or need to go this route.   You mention Arduino, we really don’t have a fully baked Adriano solution but we do have a very good solution for the RasberryPI.  Please take a look at this solution and see if it will work for you.  http://razberry.z-wave.me/  If you are dead set on Adriano then starting with the Z-Way middleware will save you the whole Sigma Designs developers kit step.

Wow!  Too serious of a commitment and way to expensive for me…I have seen this strategy with what I consider low level technologies before.  However, there is an opportunity to become the 802.11 of home automation/sensor networks.  It is not clear to me that just because I can buy products in Staples that use a wireless technology that it will become the dominant standard.  We saw this with TCP/IP (in some ways versus Novell”s IPX/SPX).  The openness of TCP/IP – as well as its ability to work well across routers across long distances – provided an environment where solutions thrived.  Z-Wave’s strategies (and others similar – or others at a higher price point during exploration stages) sadly preclude these technologies.

Which leads me to think of my exploration of RF for home networking as the story of Goldilocks and the three RF frequencies….The first I tried was the nRF24L01.  I found this solution to be unreliable and not well supported or embraced by the community.  Of course, reliability doesn’t improve since I purchased the chips on eBay.  I found the TX/RX to work intermittently.  The software library was not of the ease of use and quality.   Also, it was not obvious how to participate in a community of folks using them.  I would scurry to different forums and websites, with no dominant forum to absorb the findings of the folks that have come before me.  Thus, it was hard for me to ask questions and get timely and to the point responses.  I’m also concerned with this RF using the 2..4GHz frequency.  We have a lot of BlueTooth devices.  Plus a higher frequency will not have as far a range and will also use more power.  I’ve decided against using the nRF24L01 moving forward.

The second RF I tried was the JeeNode from JeeLabs.  There are many things to like about JeeLab’s products.  A LOT of GREAT THINKING went into different aspects – such as power consumption and RF Frequency.  The library is robust.  There is many postings of why different components are used and details on the software library.  The software library is robust.  The JeeLabs forum is active, extremely knowledgable, and helpful.  The challenges I have include I am not bought into the port abstraction for attaching sensors.  It is easier for me to think in terms of the Arduino pins and not a node representation.  I got spoiled by the DHTxx library that was available for the DHT air temperature and humidity sensor.  I wanted such a library for all the sensors.  Alas, not all the sensors use I2C not have as nicely done libraries.  However, the sensors are extremely simple with extremely simple APIs – so not having the simplified   Also, the RFM12B is an old chip that has been replaced.  I would like a chip that I can get from multiple sources and is not generally discontinued.  Yes, Modern Devices can still get chips based on their agreement with Hope RF.  But it is still a chip that the manufacturers no longer wish to produce.

The third is the one I am using for this post, the 433MHz TX/RX components.  These chips are great for initial experimentation because they are extremely simple to use.  There are significant drawbacks as my knowledge improves on indoor RF solutions.  I found the range to be inadequate.  Either due to RF interference or range, there were many times when I was lucky to get one reading after a minute when sending readings every second.  There was one wall and about 50 ft between the transceiver and receiver.  The library is very basic and extremely simple to use.   There is no support for very helpful features like RSSI feedback to help set up the best reception experience.

The most promising looks to be Hope’s replacement for the RFM12B, the RFM69.  LowPowerLabs looks to have promising “kits” that have been built on the shoulders of the great work JeeLabs has done with their JeeNode.

Transmitting Sensor Data

As you can see from the image of the current implementation of a hydroponics sensor node:

 

hydroSensorNode

 

(…I’ve got to start cleaning up the wires, shrinking its size, put it into a waterproof container, and figure out where to put it so that it doesn’t interfere with the plants!  I haven’t done this so far because I am test, adding, and removing parts.  For example, this version includes a 2×16 LCD that is an immense debugging help.)

The Base System will be receiving sensor readings on air temperature, humidity, water temperature, lux, color temperature, and pH.  I’ve discussed each of these in earlier posts.  Currently what is transmitted is “hard coded” such that the Base System is expecting a packet that contains these readings – and no other kind of packet.  In the future, there will be at least one other type of packet – the header packet which tells the Base Station what sensor readings will be sent and in what order.  For now I went for the “quick and dirty” approach to gathering data.  Also in the future I will support multiple nodes.

As shown in SensorNode433.ino:

  • I take a reading every 5 minutes.  Each minute I display the number of minutes that have passed since the last reading on the LCD.Time can be adjusted by changing TIME_INCREMENT_DELAY – the time between updating the number on the LCD and NUM_TIME_INCREMENTS – the number of increments before a reading + 1.  Thus, TIME_INCREMENT_DELAY*(NUM_TIME_INCREMENTS-1) = time between readings.  
  • the first byte sent over is the number of sensor readings that will be sent.   This will be used by BaseSystem433.ino  to determine if all the readings are contained in the received packet.
  • all readings are sent in uint16_t format.  

 

Here is a line of sensor readings:

Sensor

DEC

HEX

Humidity

43

2B

Air Temperature

68F

44

Water Temp

66F

42

Color Temp

5634

1602

LUX

30000

7530

pH

61 (6.1)

3D

The packet sent over the wire is 13 bytes.  The contents of each byte given the sensor readings is:

6

0

2B

0

44

0

43

16

2

75

30

0

3D


Keep in mind that Arduino is little endian.

As I used in previous posts, I will use the:

I am not yet comfortable with the pH readings since I do not calibrate as often as I need to.  In the future I will add this functionality using a technique similar to the one shown in the Sparkfun video.

 Since my post on the need to conserve SRAM, I am more conscious of saving SRAM.  I:

  • put the freeRam() function into a library called debugHelpers (TODO: GITHUB LINK) so that I can use this handy function to check the remaining SRAM from within any sketch.
  • started to standardize on using Serial.print(F(” “)); so that debugging strings go into Flash and not SRAM.
  • dropped using the  Dallas Temperature Control Library.  I used the OneWire library instead.  I feel much more comfortable with Arduino programming and always find it best to get rid of abstraction layers when possible.
  • Avoided using String objects.  While the flexibility and additional functionality are useful, as David Crocker points out in his blog post, “Five things I never use in Arduino projects”, using String objects come at the cost of dynamic memory allocation and additional memory overhead.

With all sensor readings except the pH reading, values can be returned as uint16_t.  The pH value is different because at a minimum the first decimal place is needed. The pH reading comes in as a string such as “7.45”.  I convert this to an integer that puts the first decimal place in the single digits space and the integer value(the value before the decimal point) in the tens and hundreds place.  For example, 7.45 becomes the integer value 74.  Doing this makes it easier on sending and receiving the packet at the expense at the “goo” needed to map into pH readings.

I added an LCD monitor to the Hydroponics Sensor Node.  This way I could write debug statements to the LCD while the sensor node was running near the hydroponics station.

Receiving Sensor Data

This configuration of the Base System includes an Arduino Uno, the SD Shield, and the 433MHz transceiver.

BaseSystem

The configuration plus the additional memory used by the BaseSystem.ino sketch left 595 bytes.   I will definitely have to move to an Arduino that has more than 2K SRAM.  The most likely replacement will include the ATmega 1284P.  No longer does the Base System need to feel closed in because of too little SRAM.  The ATmega 1284P has (a whopping compared to the 2K of the ATmega 328P!) 16K of SRAM.

Viewing Sensor Data

There is code in the BaseSystem433 sketch that checks to make sure the readings are valid, creates a string containing the date, time, and sensor readings in CSV format and then stores the string in the datalog.csv file.  Each time the Arduino is reset/starts, the earlier  datalog.csv is replaced with new readings.  Ultimately, deleting the old data is not what I want.  For now it is the easiest.  Since I know how I could implement a better handling of data logging files, I’ll leave that for something to do in the future.  

At this step I can start to analyze the sensor readings in a spreadsheet application.  This image shows a line chart of the readings for the air temperature.  There is a dip during evening hours, especially during the times when the LEDs are off.

ExcelAirTemp

 

While I took a reading every minute, the Base System did not receive all of them so the time between readings varies.  It is good enough for my purposes.   I did have to more the Base System closer to the Hydroponics Sensor Node.  As noted earlier – the 433MHz is not the one I will use in the future for the home sensor/automation network.

I have also included the .csv file in case you are interested in playing around with it.

What’s Next

  • resolve or at least gain a better understanding of the challenges I have had with components not sharing the SPI bus.  From a learning perspective, I will go a little deeper than I probably should because – heck learning is extremely motivating and fun – and it will help me better understand interactions between an Arduino and components. 
  • send the Sensor readings to the Xively service and analyze from “any device.”
  • update what is going on with the hydroponics system. 
  • finalize the home sensor/automation network RF.  
  • finalize the components of the Base System.
  • determine what probe and “stamp” to use in order to help with measuring the amount of nutrients provided to the plant’s roots.

and other things I haven’t thought to list.