Raspberry Pi Temperature Monitor, Mark II

A while ago I set up a Raspberry Pi based temperature monitoring system, to monitor temperatures for the equipment room and the server rack and freezers located in there. I still need to write a post detailing the software and hardware used for this, hopefully I'll get around to that soon. In short, I'm using a bunch of 1-wire DS18B20 digital temperature sensors connected to the Raspberry Pi. Today, though, I spent my time upgrading the case and fixation of the temperature monitor.

Mark I had a simple plastic back and a cardboard front, and was taped to the network/electricity rail in the room, like so:

While expedient to get set up and move the mess of cable out of the way, this setup has some problems. First, the Raspberry Pi tended to crash a lot. Not the software, but the hardware. That's because this yellow tape sucks, and that's why eventually I brought some proper duct tape and "fixed it". Still, it was clear that eventually even duct tape would succumb to the pull of gravity. Also, we'll lose power for a scheduled test of the emergency power system on Thursday, and I wanted to get my little Pi into the server rack so I could hook it up to the UPS.

For the Mark II case, I wanted to get a proper plastic case, front and back. Fortunately, when you work in a lab, there are a lot of petri dishes around. We also have the large square ones. Two of those, a drill, a hand saw and half an hour later, the Mark II case was ready to go.

It's attached to the rack door mesh with cable ties (yay, cable ties), and nicely out of the way of the 19" slots. Also visible is the custom board with audio jacks that the sensors are connected to. The falcon tube cap on the right is a stand-off so the add-on board won't push down too far, that still feels a bit hacked. Oh well, we're getting there.

Update 2013-09-18, 08:32 UTC: Turns out there is one unexpected feature with mounting the "in-rack" sensor on the door as well. It is now possible to track when I open and close the door, and of course the temperatures are off in that case.

The anatomy of a freezer failure

A while ago I built a system to monitor temperatures in my server and freezer room. Yes it's the same room, but I could only get the space and air conditioning once. I still need to write the blog post on how I set up the temperature monitor system, but basically it's a Raspberry Pi with a number of DS18B20 1-wire digital temperature sensors. I'm using statsd and graphite to graph the data, and a separate set of scripts to send email alerts when specific sensors go above their configured temperatures.

Today was the first time the system was put to use, and I'm pretty happy with how it all worked out. I can even reconstruct what probably happened. Looking at my graphite temperature graph for that freezer, the following story unfolds:

A bit after 11:00, someone opened the freezer. They used the top drawer, which is pretty full with boxes holding Eppendorf cups. These boxes come in a low and a high version. The inner ceiling of the freezer is lower in the back than it is in the front, and care must be taken that the last row of boxes in the drawer only consists of the low boxes. If you place a high box there, the drawer will not slide in all the way but will stick out about 5 mm. These additional 5 mm will prevent the door from creating a seal when closed. At around 11:05, the temperature control of the freezer notices that the temperature is too high and starts to cool more. Note that my own temperature sensor is sitting right in the cold air vent rigth on the top of the freezer, so it's probably measuring more extreme temperatures than the built-in sensor.
After running the compressor at full whack for about 100 minutes, the cooling finally fails and the freezer starts to thaw. At 12:45, the temperature rises above -10 °C, triggering an alert email. Thus alerted, yours truly walks over to the freezer room and notices that the door doesn't close correctly. After shuffling some boxes around, the door closes correctly again and the temperature begins to fall.
Unfortunately, the controller of the freezer must have crashed at 12:40 as well, because the freezer does not resume cooling. The rising temperature triggers a second warning email at 13:06. This time, it takes a bit longer to realize what the actual problem is about, but eventually I realize that the cooling isn't doing anything and toggle power on the freezer. The cooling comes back on, and the situation is improving again.

The first live failure has shown that for our purposes the temperature monitoring systems is working nicely. While the freezer did fail during the day, the controller got stuck somehow and didn't trigger the acoustic alarm we should have gotten in case of a failure. The email-based system works, and the response time during the day is sufficient to avoid serious problems. For a cost of about 100 EUR in electronics, the Raspberry Pi / DS18B20 combination is a cost-effective way to monitor a room full of freezers.

PhDOps is now a thing.

Since March 23rd 2013, I've been using the #PhDOps as a tag on twitter for posts relating to the work I do maintain the antiSMASH web service. As I've been collecting blog posts that I needed to write about running an academic web service while having a full time job getting a PhD, I figured this would be a good name for a blog to collect said posts.

Stay tuned for a post on using Docker to deploy Linux container (LXC) based application containers.