Ready for Christmas

One of the jobs I want to hand over to the WebCell is turning the Christmas lights & decorations on and off. I’ve been using mechanical timers, which is fine until there’s a power outage and they fall behind.

I spent three days last week creating the web pages needed to enter the Insteon control devices and the times to turn them on and off. It was very similar to the thermostat pages and I used them as a starting point. All the underlying scheduling features of the WebCell were supposed to ready, but one of the days was spent debugging new bugs the pages exposed.

I wasn’t satisfied when I was done, because I was still skipping over one of the last unimplemented features in the CellScript specification: the ability to schedule relative to sunrise and sunset. It made a lot of sense to do this now, since I would want the lights to go on about sunset. So I attacked that problem this week.

I did a web search to find the equations, and used a spreadsheet to enter test values and see all the intermediate values. The equations use a lot of trig functions, but they’re all in the C runtime library for the AVR. It took two days to get “sun-based” scheduling fully implemented, and one more day to rework the scheduling web page to accept entries. I put one set of decorations on a sunset-to-midnight schedule so I can watch and make sure it seems to be working right.

Time to Fix the DST Bug

I haven’t done any work on the WebCell since August when it took over my thermostat schedule. But Daylight Savings Time ends in a few days, so I thought I should fix a bug in the code that handles it.

I didn’t test it, but I’m pretty sure the simple way the WebCell was figuring the end of Daylight Time would behave badly. I suspect it would have spent an hour ping-ponging back and forth between Daylight and Standard Time. You can see how it would happen: at 2:00 AM it sets the clock back to 1:00 AM; then it decides Daylight Time isn’t over for another hour and sets it back to 2:00 AM, etc.

The fix wasn’t hard – just do everything in Standard Time. Instead of Daylight Time ending at 2:00 AM Daylight Time, it ends at 1:00 AM Standard Time.  All fixed. (Internally, time is kept in UTC [GMT] with time zone adjustment added to get Standard Time.)

All the parameters of Daylight Time can be changed in CellScript code, but up till now I hadn’t created a user interface to do it. I spent more time on the web page to view and edit Daylight Time settings than I did fixing the bug.

The Scheduler

When I started this project, I envisioned the WebCell as the user interface, via web pages, for otherwise self-contained home automation devices. For example, the WebCell might pass on to a thermostat a list of temperatures and times that the user had entered, and the thermostat would handle it, even if the WebCell was subsequently disconnected. In practice, however, the devices I’ve encountered need more supervision – they need a controller to maintain the schedule.

In January 2011, I started the design of the CellScript scheduler. I wanted total flexibility, and used the calendar feature of Microsoft Outlook as a guide for creating recurring schedules. The plan made its way into the CellScript Language Reference, and from there the coding began. By the end of February, the scheduler successfully “fired an event” on time – simulated, of course.

In March I felt the scheduler was in pretty good shape. It was time to write the web pages that would collect the settings and feed them into the scheduler. To help with this I took a class at the nearby community college in cascading style sheets (CSS), which is all about making web pages look the way you want them to.

Still I found writing the web pages to be a major challenge. The pages all use AJAX techniques and YUI 3 to make them as easy to use as possible. I was creating these pages to work with an Insteon irrigation controller and a pair of Insteon thermostats (for two separate furnaces in my house). For each system I had a setup page (irrigation zones, for example), and then a settings page (such as watering time for each zone and watering schedule).

My priority was the irrigation system, because I’d already wired in the Insteon controller. Fortunately we had a very cool and wet spring in Western Washington, giving me more time to get the irrigation pages ready. In late May the WebCell succeeded in starting an irrigation cycle on schedule, which turned out to be about two weeks to spare.

The push to get irrigation running on the WebCell gave me my first real experience with CellScript (as opposed to writing test cases). I did some tweaking and bug fixing before returning to the next set of web pages, for the thermostats. On August 13, 2011, the WebCell took control of the thermostat schedule. Program size is now at 83,464 bytes. And I will be taking a break from WebCell development for a while!

Slow Progress on the List

With the production prototype in hand and working, and plenty of program space available in the XMEGA, 2010 was spent methodically working through “the list” – the CellScript Language Reference. Hardware features came first, such as ADC, DAC, PWM, and input capture (measuring the pulse width or frequency of an incoming signal). This would allow testing circuitry on the DevKit whose job was to demonstrate these features.

But the list was very long, and sometimes got longer instead of shorter. Work items included getting multiple scripts to run at once, implement the DHCP network protocol, storing important data in the internal non-volatile memory (EEPROM), adding “MIME types” to web pages – and fixing lots and lots of bugs.

The WebCell has several ways to communicate with the outside world besides Ethernet, including three serial ports, an SPI (Serial Peripheral Interface) port, and an I2C (Inter-Integrated Circuit) port. “The list” has just a few functions and properties that CellScript needs to utilize these ports – things like read(), write(), and bitRate. Implementing these for the WebCell was a clearly understood task. However, what would these do on the simulator? This question applied to all the hardware features. I spent much of August creating ways for the user to interact with simulated hardware features, allowing entry of the stuff going in, and display of the stuff coming out.

In September I started creating web pages designed to demonstrate and test hardware features. In the process I looked into YUI, the Yahoo! User Interface. The first page I made was for testing PWM (pulse-width modulation), and it used a YUI “slider” to adjust the PWM width. YUI also made it easy to have the web page contact the server in the background (in this case, the WebCell or simulator) for instant updates – a common way to make web pages more interactive called AJAX (asynchronous JavaScript and XML).

The demo web pages were helpful in completing testing of the production prototype. Satisfied that I’d found all the wiring errors, I began revision of the PCB. On September 16, 2010 I sent in the order for new PCBs, and received them October 4. When assembled and put to the test, no problems were found in the hardware design. I now had two complete working units: the production prototype, with its cuts and jumpers; and the final production model.

Using the new hardware, I set out to give the WebCell the ability to have its firmware updated over the network. Part of that process included transmitting a somewhat large firmware file from the PC to the WebCell via FTP, which turned out to expose numerous problems in previously untested FAT file code. I had had a similar experience back in July, which had got me thinking about how to test the FAT file code better.

The CellScript simulator was a great way to test and debug CellScript itself, because the same source code was used for both the simulator and the real WebCell. However, the simulator just used Windows for file operations, so it didn’t help test the FAT file system. But I realized I could change that, using raw sector read/write calls through Windows. In just a few days I was able to create a special version of the simulator, just for testing, that used my FAT file code to read & write files on an SD card on the PC.

I did a lot of testing and fixed a lot of bugs in the FAT file system, now using nothing but the simulator. In December I ended up doing a major overhaul of the whole system. And now I had a large set of tests I could run anytime to make sure the FAT file system was working right. Program size when I was finished was 70K, out of the 128K available.

Final Hardware Design

In early 2008 Atmel announced a whole new line of microcontrollers called XMEGA that sounded pretty interesting. Compared to the existing ATmega line, they offer much higher performance and much more capability, all at around the same price or less.

As a new product, availability is an issue. I originally placed an order when they first showed up in my distributor’s catalog in September 2008, and just let it ride on backorder. The order was eventually cancelled because that specific part (ATxmega64A4) was never produced. A year later I did get my hands on one of the top-of-the-line XMEGA chips, the ATxmega256A3. This was not the chip I would use for my final hardware design, but it did give me a chance to test it out.

The XMEGA family is only available in surface mount (SMD) packages, so I mounted mine onto a DIP adapter. Then I put it on my solderless breadboard in place of the ATmega324P, wired up to an SD memory card socket and serial port. By the end of September 2009 I had my “mini-DOS” FAT/SD test program working on it.

SD Card Breadboard with XMEGA. Parts include a small 3.3V regulator, reset button, and simple RS-232 serial interface just using a 74AC04 inverter.

Also in September I started work on the hardware design for the “production” WebCell, using the ATxmega128A3. I had never used either of the radio circuits on the engineering prototype, so I decided to leave them out. In the home automation application I envisioned, the WebCell would have to connect to some other device. I had experimented with Insteon products so I knew the WebCell could be hooked up to control their devices, for example.

In November I finished the layout of the WebCell PCB. Next I started design of the “Development Kit”. This is another PCB that the WebCell can plug into that makes it easy to try out. For example, it powers the WebCell from USB, provides a serial connector, and includes a reset button.

I also designed a variation of the WebCell that I call the FileCell. This variation removes the network interface and replaces it with a USB port and battery backup. So instead of being a web server, it is more of a general-purpose controller module. It could be particularly useful for data logging because of the high storage capacity available on the microSD card. I wasn’t sure if I would ever build this variation, but I wanted to explore how it would be different from the WebCell so I could make sure the Development Kit would work with both. Because of the battery backup, I did end up changing some of the interface connector pins on the WebCell.

I had to order samples of various parts to decide which ones I wanted to use and how they would fit. I finally placed orders for production prototype PCBs on January 21, 2010. I had been planning to get PCBs for all three designs, but in the end I saw no reason to build the FileCell variation yet.

The PCBs arrived on February 8. I took them and a kit of parts to my local electronics assembly company, Printed Circuits Assembly. Both boards use chips in leadless (QFN) packages, which are really hard to solder by hand. I got the assembled units back about a week later.

WebCell on DevKit. Blue jumper wires correct some PCB layout errors, and provide test points to internal circuitry. Long tails on board-to-board connector were useful as additional test points. Micro USB connector for power is in lower right corner.

It took a few weeks to get the software ready for the production prototype and to get the prototype working. The final hitch was an error in the wiring of the Ethernet jack, requiring cuts and jumpers of the PCB traces. Once that was fixed, on February 28, 2010, the prototype began serving up web pages.

During the wait for assembly, my work on the WebCell software started bumping up against the 64K program limit of the engineering prototype. The production prototype with its 128K program size came just in time to keep me working.

WebCell Script File Updates

I spent the spring of 2009 working on my implementation of the FAT file system. A year earlier I had it reading files from an SD card, but now I wanted it to write to the SD card. Before the start of summer I had the essentials working – creating folders, creating files, and writing data to files.

This gave me the opportunity to relieve a long-standing annoyance. Up to now, when I wanted to test a new or updated script on the WebCell, it took a few steps:

• Remove the microSD card from the WebCell.
• Put the microSD card in an SD adapter.
• Plug the SD adapter into an SD slot on my monitor.
• Copy the test script from the PC to the microSD card.
• “Eject” the microSD card.
• Remove the SD adapter from the monitor.
• Move the microSD card from the adapter back to the WebCell.

Now that the WebCell could write files itself, this process could be eliminated by sending the file to the WebCell over the network connection. And the way to do that was with FTP – File Transfer Protocol. I set to work on the implementation.

On July 9, 2009, I used FTP on Windows Explorer to copy a test script from the PC to the WebCell. This was a satisfying if not momentous milestone. Program size on the WebCell with FTP completed was at 55,510 bytes out of the 65,536 bytes available.

With the WebCell receiving files by FTP, I felt obligated to add an FTP client to the CellScript simulator. The simulator allowed me to edit scripts, so it just seemed wrong if it couldn’t send those updated scripts on to the WebCell.

Unfortunately, this would mean a lot work to do on the simulator. The simulator was written in C++ so it could share the code for CellScript with the WebCell. But I really missed the very elegant and refined environment that came with programming in Microsoft’s C# programming language. I decided I wasn’t going to take it anymore.

I split the simulator in two: a module in C++ that carried the shared CellScript code, and a module in C# that handled the user interface through Windows. Once I found out how easy it was to get the two modules to talk to each other, I was kicking myself for not doing it this way in the first place. The split and translation of the UI from C++ to C# took about a week.

Once the new simulator was working the same as before the split, I started the job of adding FTP. I created a “File Manager” similar to a stripped-down Windows Explorer. But it tracked both script files on the PC used for the simulator, and files on the WebCell using FTP. It then presented them in an integrated list showing what files were different between the two. The finishing touch was that it would monitor when changes were made to simulator script files, and could automatically copy them to the WebCell immediately.

Just What is CellScript?

I had jumped in and started implementing CellScript using a book on JavaScript I’d bought and the ECMAScript Language Specification Edition 3 as my guides. But after I had some of the basic language features working, I needed to give myself more concrete direction – it was time to define just what would be in CellScript.

First of all, CellScript is only a small subset of JavaScript. All I wanted were the basic features that any programming language has: evaluating expressions, conditional (“if”) statements, looping constructs, etc. I didn’t want to try to implement any of the special features of JavaScript. I needed to draw the line at what was in and what was out.

Second, CellScript needs to extend JavaScript so it can access and control features of the WebCell. This is done by adding “host objects”, such as the Net object for dealing with the network interface or the File object for managing files on the SD card.

In late February 2009 I created the CellScript Language Reference. My productivity had started to lag as I paused each time I finished a feature to figure out what was next – or worse, spun my wheels. Once the spec was written, I was able to charge ahead.