Enginursday: Troubleshooting a Simple Problem
Sometimes troubleshooting requires you to think both literally and figuratively "outside the box."
Hearing about frigid temperatures in the Northeast recently got me thinking about how I used to have to troubleshoot static problems in a previous job when the weather was consistently cold and dry. Most of the time the problems were quick to identify, and easy to fix. But there was one time where it was a simple problem to fix, but hard to spot.
Backstory
My first job out of college was as a Junior Engineer for a large printer manufacturing company. The department I was in focused on the specialized sensors used in the printers. The sensors ranged in complexity from simple off-the-shelf temperature and humidity sensors, to custom imaging modules that were built in-house (similar to the imaging sensor used in a scanner). The imaging module we were investigating is similar to a competing imaging array made by Mitsubishi, shown below.
As a Junior Engineer, I mostly worked with a Principle Scientist who had been with the company for 30+ years. His background was the same as mine (B.S. in Electrical Engineering), but he specialized in electrostatics (static electricity). Most of the year we focused on the Xerography of the printers --- moving toner from the cartridge or bottle to the paper using static electricity. But every year on the coldest, and driest, days of winter, we would get calls about static causing problems on a machine. Most of the time the problem could be fixed by installing an antistatic brush in the paper path before where the problem was noticed, and it would go away.
One time, though, we were left scratching our heads for a couple of days. The printer we were dealing with wasn't a personal desktop printer, but an industrial printer that is almost 7ft tall, and a little under 20ft long --- printing on a continuous sheet of paper up to 500ft per minute. The paper is unrolled on one side, printed, flipped and printed on a second printer before being rolled back up, or folded and cut to size. Thankfully, to get from the front of the machine to the back you didn't need to walk all the way around the printers; the box that flipped the paper was made of steel and allowed you to safely step over the paper to the other side of the machine.
The Problem
The reason we were called in was that, after a period of time, the imaging sensor that my group was responsible for would reset itself and cause the machine to stop. We stopped by and asked the operator of the machine to describe the problem and try to replicate it. We were quickly able to figure out that the problem was static-related and told the machine's operator to install the antistatic brushes in the paper path both before the sensor as well as after, and the problem should go away. Spoiler: it didn't.
The next day were back at the machine making sure the brushes were installed correctly (which, of course, they were) and did even more head scratching. We started to look at the second machine to see if somehow the static was building up on that machine and getting to the first. But before we ventured too far down that rabbit hole, we wanted to check the steel box that flipped that paper over, just to do our due diligence and eliminate it as a factor. We noticed two things odd with the box. First, there was an ethernet cable connected to the problematic machine on one end and tossed under the box on the other. The second problem was that the steel box should have had a grounding wire on it to prevent static buildup on the paper as it passed from one machine to the other, but it didn't.
As we moved onto the second machine, we asked if the cable needed to be connected to the first machine, and to remove the tripping hazard if it didn't. For the sake of thoroughness, we asked to install a grounding kit to the box to avoid potential future static problems on the second machine. We continued with our testing but couldn't replicate the problem anymore. As it turns out, the problem was that, as the paper moved on the rollers in the box, it was acting like a Van de Graaff generator until it arced over to the ethernet cable. The static charge was large enough that it interfered with the image module, causing a fault, which was recovered by a reset of the module. In a more humid environment any other time of the year, the problem never would have presented itself. But as it was, the problem managed to take multiple engineers multiple days to figure out.
In a complex system like a printer, sometimes it's the simplest of reasons that cause the largest headaches. What problems have you had to troubleshoot that took you longer than you'd like to admit to figure out? Let us know in the comments!
