Saturday, March 27, 2010

The Importance of Qualitative Research

For five years I worked as a Quality Assurance Technician/Specialist for a major corporation, before I took time off to be a stay at home mom. Most of my work revolved around routine quantitative tests and research at the national QA lab. However, I had the opportunity to work on some investigative research during that time. One nice thing about doing internal research for a major corporation is that you don't really have the option to put a spin on your test results. There is always an external bottom line that you are accountable to. While in the academic and sales world, some researchers can talk circles around their results and still retain their positions, in the quality assurance world, if your research is faulty, it shows in the product and in the accounting. Faulty quality research costs a company money and can get you fired.

During my time in the QA field, I learned several things about research. First off, it's extremely easy to end up measuring the wrong thing. I cannot emphasis that point enough. During my first year, I was the technician for a new engineer, and he and I found this out firsthand. We even had a strong disagreement over one test on this matter. I noticed that there was something weird going on with my test results and brought it to his attention. He wanted to wave it away, because my results were confirming his hypothesis. If there is one thing I hate doing, it's fudging data. So I told him flat out that it may prove his hypothesis, but as a technician, I was not going to stand by the results. He finally told me that I could repeat the test. I'm sure he was expecting me to back down because he knew I hated doing that particular test, especially for the number of repetition he was asking for. Instead I agreed and then after clarifying that I would be doing the test the same way as I had the first time, I asked if I could switch the order of the samples. He agreed and the next set of results showed that I was right. To the engineer's credit, he then accepted that he was mistaken, listened to my observations, and figured out what actually was going on - and was able to confirm it. A few years later, he borrowed me back from the manager I was then working with, because he had a $5 million bizarre problem to solve and he wanted a technician who would tell him if things were looking screwy.

Between those times I worked for that one engineer, I worked for a manager who had been a university professor at an engineering school before working for the company. I was given the task to compare two types of color measuring machines. (Since, to the best of my knowledge, no one reading this blog is in the flexible packaging field, I'll dispense with the technical names.) The newer one was developed for the auto industry and we needed to see if it could meet our needs. Before I could even begin to compare the two machines, I had to do a search of the literature before I could even have an idea what I was actually testing. I also talked to our R&D packaging scientist and the manufacturer's representatives. I even resorted to going through my mother's old books on painting with color, which provided me with a lot more on the subject than I originally expected.

In short, under the direction of my manager, I did qualitative research before I started anything quantitative. That's because it's the qualitative research that shows you what you should be measuring. Most people don't understand that. When I took the research methods class for my masters, I really had to bite my tongue not to go into an impassioned rant over the necessity of qualitative research as a precursor of decent quantitative research. The R&D scientist I worked with for the color project was brillant and a good deal of his brillance came from the fact that he did his qualitative research before he launched his quantitative research. In the quality assurance world, measuring the wrong thing costs money and jobs. There's no time for weak research based on "well, this sounds like this might be the cause." You need to have solid reasons for your choices before you can even start your tests and in the corporate world, those choices are challenged more stringently than what I've seen so far in the academic world. Of course, this may not be the case on the doctorate level. I hope that is the case when I go back to school for my doctorate, because through my quality assurance training I have developed an obsession with doing thorough background research before I develop a test procedure. This is the reason my capstone is basically a literary review, because I would rather do a thorough literary review than a half-thought out series of qualitative tests.

Actually, I'm not sure I can even bring myself to do testing before doing a literary review. You see, my last job with this company was manufacturing defects coordinator. I was transferred to a plant to help them reduce their packaging defects. The plant management was convinced that the suppliers were sending them inferior flexible packaging. I spent the first few months testing roll stock to confirm that this plant was getting the same quality of packaging that all the others were. The plant manager didn't take that very well and I was told that I had six months to find out what the problem was or my job would be eliminated. For the first two or three months, management kept giving me things they wanted me to test for them as possible causes for their high defect rate. Out of pure preservation, I began to study other possible contributing factors. I used my connections with the national lab to talk to the engineers who trained the trainers who taught the packaging machine operators how to do their jobs. These were also the same guys who followed up on the quality of the machine maintenance people. They assured me that the trainers and the plant maintenance people where I was at were top notched. I talked with the trainers in the plant and made friends with the maintenance guys. It was obvious from their dedication and knowledge that they were, indeed doing their part. It was also obvious through the observed quality of their work, which they gladly let me examine.

I knew we were all missing something, but I couldn't figure out what it was while doing all the tests to make management feel better. So, I took a stand and reminded them that they brought me there because I was a specialist and if they were going to make me accountable for this, then they needed to let me do the job I was brought in for. Again, the plant manager was not impressed, but I figured I was doomed anyway with the way things were going, so I stood my ground. He said he wanted me to do line audits. In the past, I had been doing warehouse audits because no one wanted me to interrupt their production flow. I told him that I would do it on the grounds that I created my own audit criteria. To make everyone happy, I put in every common packing flaw that could be seen by a non-destructive inspection. However, well aware that I still had no idea what the real problem was, I made sure I made a space on my audit forms for comments - just in case I stumbled across a clue or two.

Then I started my line inspections. I hadn't even finished my first bag inspection, when the packaging machine operator came up to me and asked in a surly voice if he was going to see my results right away or was he going to have to wait until his manager yelled at him about them. Dumbfounded, I said, "You don't get my reports?" After he confirmed that was the case, I asked him if he wanted me to make him a copy and bring it back to the line as soon as I finished with everyone. He agreed. During this time, a light went off in my head, as I reviewed the stories I knew about how my dad and maternal grandfather managed their employees. The packaging machine operators weren't getting enough feedback. And if they weren't getting enough feedback, then chances were, based on the first guy's comments, they weren't getting any positive feedback at all. So, in my "comment box" I mentioned specifically what was right about the bags I auditted.

I asked every machine operator if they wanted my reports directly from me. All of them said yes. I instituted what I called my positive feedback program. Sometimes, before I even started checking for flaws, I would write down what looked good on the bag. After a week of this, the machince operators started looking forward to my audits and asked for my opinion on some of their issues. During the third week, I came across a perfect bag. I asked the machine operator if I could display it in the lunchroom with his name attached to it. That bag was followed by others from other machine operators. Three months from the start of my audits and positive feedback, the manufacturing defects numbers had dropped 70%.

It wasn't the quantitative studies that found the issue. It was the qualitative investigation that did the job. The quantitative studies only backed it up. International Organization for Standardization has proceedures it dictates for those companies wishing to be ISO certified. I've actually taken classes on ISO certification a few years back and I can assure you that their quality standards take into account the qualitative aspects of processes too. I'm tempted to pull out my books and give examples, but I think I'll save that discussion for another post.

No comments: