A Change of Schedule

Wow, it’s been a long time since my last update. I have continued to work on developing the WebCell into a product. I’ve used a few contractors from Elance.com to help with specific things, like make icons for the CellScript Simulator. But I’ve spent the most time on a total rewrite of the web pages for home automation.

The home automation web pages are no longer generated by running CellScript code on the WebCell. They create their content themselves with JavaScript by using a web service written in CellScript on the WebCell as the source of data. The web service provides an interface to the WebCell scheduler, which tracks things like thermostat settings and irrigation times.

With this rewrite I also recently completed a web page that for the first time provides access to a feature that the scheduler has had from the beginning – the ability to schedule a change of schedule. Here’s how I put it to use:

I have thermostat schedules called “Home” and “Away”. The Home schedule changes the temperature setting each morning and evening; the Away schedule is basically set to turn the system off, with the heat set to 55° and the cooling set to 85°. I was leaving town for a week, but my son-in-law was staying at my house for an additional night, so I set the Away schedule to start at 8AM the next day. I also set the Home schedule to start in the morning on the day I get back. So now I get to save energy when nobody is in the house, but it will be a comfortable temperature when I return.

Is It a Product?

I recently made a revision to to the Development Kit PCB, and I’ve been testing out the revised prototypes. Everything is looking good so far. I didn’t need the circuit changes for my own use -– at worst, I could cut-and-jumper any of the PC boards I would want. This is the latest in a string of actions that only make sense if I turn the WebCell into a product.

Earlier in the summer I spent a fair amount of time creating simple web pages to demonstrate the I/O capabilities of the WebCell. Then I started writing a manual (a scary big project!). None of these things are stuff I would need to do just to keep using it around the house, but I could chalk it up to keeping myself occupied.

That changed when I decided to spend money on product development. Paying to have a new PCB fabricated and assembled is one example. I also upgraded my version of Help & Manual to make writing the documentation easier.

Yes, I would like to make the WebCell into a product if I can tackle all the hurdles. At this point I don’t know that I can, because what needs to be done isn’t what I’m good at. Writing documentation. Creating more web pages. Making the web pages I have look professional. Maybe I will find a contractor to help out (spending money again!).

It’s About Time

I happened to check the time on the WebCell the other day and noticed it was running about 4 minutes fast. Currently the WebCell clock only sets itself (using a network time server) when it powers on, so that error would have accumulated over the few months since the last winter power outage. I guess it’s time to change that.

Timekeeping on the WebCell originates with a 25 MHz crystal on the Ethernet chip. The Ethernet chip divides it by 4 to 6.25 MHz before sending it over to the AVR microcontroller.  The AVR multiplies it by 5 to 31.25 MHz, very close to its max clock speed of 32 MHz. A counter in the AVR divides this by 62,500 resulting in 500 Hz, which is then used for all internal timekeeping.

The crystal is supposed to be accurate to 50 parts per million, which can also viewed as one part per 20,000. Since there are 60 * 60 * 24 = 86,400 seconds per day, the crystal could have a max error of 86,400 / 20,000 = 4.32 seconds per day. Even resetting the time every day, that’s a little more error than I’d like to see.

The number 2^16 = 65536 is a bit special in programming, and I chose it as the interval in seconds between resetting the clock. This works out to about 18 hours. Not only that, but if the clock is off by one second or more when it gets reset, the counter that puts out the 500 Hz gets its divisor adjusted by one tick. So if the clock was running fast, the counter will be adjusted to divide by 62,501. If the clock continues to show an error at subsequent resets, the divisor will continue to be adjusted, one tick at a time. Given the accuracy of the crystal, this adjustment shouldn’t get larger than 3 or 4 ticks off the nominal 62,500, and I’ve set a limit of 20. This adjustment is maintained in the (non-volatile) EEPROM so it won’t be lost on power failure.

With that done, I decided to tackle another time issue that’s been bugging me. When I copy new web pages to the WebCell, the date/time stamp on the file is not copied with it. Instead, the new files are stamped with the current time. This can get pretty confusing when you look back and have to realize that a file time stamp on the WebCell that is after the file time stamp on the PC probably means they’re the same file.

Fixing this required hitting work items at several different levels. First, my implementation of the FAT file system did not yet support setting a file time stamp to an arbitrary date/time. Next, because these files are transferred using FTP, I had to implement the FTP extension MFMT, which sets the time stamp on a file. Finally, I had to have the FTP client in the CellScript simulator use that FTP extension when it copied files.

Fixing these time issues resolved all my immediate concerns for the WebCell itself, giving me some sense of completion.

Ready for Christmas

One of the jobs I want to hand over to the WebCell is turning the Christmas lights & decorations on and off. I’ve been using mechanical timers, which is fine until there’s a power outage and they fall behind.

I spent three days last week creating the web pages needed to enter the Insteon control devices and the times to turn them on and off. It was very similar to the thermostat pages and I used them as a starting point. All the underlying scheduling features of the WebCell were supposed to ready, but one of the days was spent debugging new bugs the pages exposed.

I wasn’t satisfied when I was done, because I was still skipping over one of the last unimplemented features in the CellScript specification: the ability to schedule relative to sunrise and sunset. It made a lot of sense to do this now, since I would want the lights to go on about sunset. So I attacked that problem this week.

I did a web search to find the equations, and used a spreadsheet to enter test values and see all the intermediate values. The equations use a lot of trig functions, but they’re all in the C runtime library for the AVR. It took two days to get “sun-based” scheduling fully implemented, and one more day to rework the scheduling web page to accept entries. I put one set of decorations on a sunset-to-midnight schedule so I can watch and make sure it seems to be working right.

Time to Fix the DST Bug

I haven’t done any work on the WebCell since August when it took over my thermostat schedule. But Daylight Savings Time ends in a few days, so I thought I should fix a bug in the code that handles it.

I didn’t test it, but I’m pretty sure the simple way the WebCell was figuring the end of Daylight Time would behave badly. I suspect it would have spent an hour ping-ponging back and forth between Daylight and Standard Time. You can see how it would happen: at 2:00 AM it sets the clock back to 1:00 AM; then it decides Daylight Time isn’t over for another hour and sets it back to 2:00 AM, etc.

The fix wasn’t hard – just do everything in Standard Time. Instead of Daylight Time ending at 2:00 AM Daylight Time, it ends at 1:00 AM Standard Time.  All fixed. (Internally, time is kept in UTC [GMT] with time zone adjustment added to get Standard Time.)

All the parameters of Daylight Time can be changed in CellScript code, but up till now I hadn’t created a user interface to do it. I spent more time on the web page to view and edit Daylight Time settings than I did fixing the bug.

The Scheduler

When I started this project, I envisioned the WebCell as the user interface, via web pages, for otherwise self-contained home automation devices. For example, the WebCell might pass on to a thermostat a list of temperatures and times that the user had entered, and the thermostat would handle it, even if the WebCell was subsequently disconnected. In practice, however, the devices I’ve encountered need more supervision – they need a controller to maintain the schedule.

In January 2011, I started the design of the CellScript scheduler. I wanted total flexibility, and used the calendar feature of Microsoft Outlook as a guide for creating recurring schedules. The plan made its way into the CellScript Language Reference, and from there the coding began. By the end of February, the scheduler successfully “fired an event” on time – simulated, of course.

In March I felt the scheduler was in pretty good shape. It was time to write the web pages that would collect the settings and feed them into the scheduler. To help with this I took a class at the nearby community college in cascading style sheets (CSS), which is all about making web pages look the way you want them to.

Still I found writing the web pages to be a major challenge. The pages all use AJAX techniques and YUI 3 to make them as easy to use as possible. I was creating these pages to work with an Insteon irrigation controller and a pair of Insteon thermostats (for two separate furnaces in my house). For each system I had a setup page (irrigation zones, for example), and then a settings page (such as watering time for each zone and watering schedule).

My priority was the irrigation system, because I’d already wired in the Insteon controller. Fortunately we had a very cool and wet spring in Western Washington, giving me more time to get the irrigation pages ready. In late May the WebCell succeeded in starting an irrigation cycle on schedule, which turned out to be about two weeks to spare.

The push to get irrigation running on the WebCell gave me my first real experience with CellScript (as opposed to writing test cases). I did some tweaking and bug fixing before returning to the next set of web pages, for the thermostats. On August 13, 2011, the WebCell took control of the thermostat schedule. Program size is now at 83,464 bytes. And I will be taking a break from WebCell development for a while!

Slow Progress on the List

With the production prototype in hand and working, and plenty of program space available in the XMEGA, 2010 was spent methodically working through “the list” – the CellScript Language Reference. Hardware features came first, such as ADC, DAC, PWM, and input capture (measuring the pulse width or frequency of an incoming signal). This would allow testing circuitry on the DevKit whose job was to demonstrate these features.

But the list was very long, and sometimes got longer instead of shorter. Work items included getting multiple scripts to run at once, implement the DHCP network protocol, storing important data in the internal non-volatile memory (EEPROM), adding “MIME types” to web pages – and fixing lots and lots of bugs.

The WebCell has several ways to communicate with the outside world besides Ethernet, including three serial ports, an SPI (Serial Peripheral Interface) port, and an I2C (Inter-Integrated Circuit) port. “The list” has just a few functions and properties that CellScript needs to utilize these ports – things like read(), write(), and bitRate. Implementing these for the WebCell was a clearly understood task. However, what would these do on the simulator? This question applied to all the hardware features. I spent much of August creating ways for the user to interact with simulated hardware features, allowing entry of the stuff going in, and display of the stuff coming out.

In September I started creating web pages designed to demonstrate and test hardware features. In the process I looked into YUI, the Yahoo! User Interface. The first page I made was for testing PWM (pulse-width modulation), and it used a YUI “slider” to adjust the PWM width. YUI also made it easy to have the web page contact the server in the background (in this case, the WebCell or simulator) for instant updates – a common way to make web pages more interactive called AJAX (asynchronous JavaScript and XML).

The demo web pages were helpful in completing testing of the production prototype. Satisfied that I’d found all the wiring errors, I began revision of the PCB. On September 16, 2010 I sent in the order for new PCBs, and received them October 4. When assembled and put to the test, no problems were found in the hardware design. I now had two complete working units: the production prototype, with its cuts and jumpers; and the final production model.

Using the new hardware, I set out to give the WebCell the ability to have its firmware updated over the network. Part of that process included transmitting a somewhat large firmware file from the PC to the WebCell via FTP, which turned out to expose numerous problems in previously untested FAT file code. I had had a similar experience back in July, which had got me thinking about how to test the FAT file code better.

The CellScript simulator was a great way to test and debug CellScript itself, because the same source code was used for both the simulator and the real WebCell. However, the simulator just used Windows for file operations, so it didn’t help test the FAT file system. But I realized I could change that, using raw sector read/write calls through Windows. In just a few days I was able to create a special version of the simulator, just for testing, that used my FAT file code to read & write files on an SD card on the PC.

I did a lot of testing and fixed a lot of bugs in the FAT file system, now using nothing but the simulator. In December I ended up doing a major overhaul of the whole system. And now I had a large set of tests I could run anytime to make sure the FAT file system was working right. Program size when I was finished was 70K, out of the 128K available.