Subconscious Information Processing OR How I Think About Problems

This post originally appeared July 25, 2011 on

When I was in 8th grade, I participated in an program called Future Problem Solving. Teams from many different schools would gather in a room, the proctors would pass out a short description of a problem and each team would have a couple hours to come up with a set of solutions, weigh pros and cons of each, then recommend a solution for implementation. We would write all of that up into a two page summary, turn it in and wait a couple hours for the grading to come back to see which team won the round.

Given that I’m a Business Analyst by trade, it probably isn’t surprising to any of you that I really enjoyed Future Problem Solving.

What had never occurred to me until I started this post is how much the habits I learned doing Future Problem Solving really fit into how I work today. See, what I haven’t told you yet is that in the months prior to the sessions, the team was given high-level categories from which our question for the round would be selected. During those intervening weeks, the team spent its time researching and studying anything we could get our hands on about those subjects. I even remember our coach going so far as to figure out ways to get copy protected material that couldn’t be sent through a copying machine to work on one anyway, just so we could all share the material.

But after we studied our heads off, we did something interesting… we did nothing. That’s right, once we read and memorized all the information we could, we took a break. For the days just prior to the event, we really didn’t do that much work. We just let the information percolate in our minds, ideas quietly bouncing off one another, marinating just long enough to produce something of quality.

So when I read about the idea of subconscious information processing, I realized that while the term might be relatively new to me, the concepts went back to some of my oldest memories. I do this all the time, for just about every project. Its routine to hear me say, “That’s interesting. I’m not sure what to do with it or how to tackle it, but let me think about it a while and get back to you.”

See, I’m all about up front research. Let me give you an example that happened just today. About 7 months ago, when I was given charge of our company’s QA functions, I started doing a lot of research about testing processes and automated testing tools. During that research, I ran across the concept of writing software that gathers a large, some might say obscene, amount of metrics data from your application. The idea is to capture and measure everything, then to graph it so people can interpret it easily. When you start, you measure everything you can think of. Then you go back and add the things you never thought of measuring. Once you hit your stride, even modest sized applications can be measuring thousands of data points, everything from user interactions to system performance to usage duration.

I spoke of the idea of using this type of metrics data for an upcoming project, but only dropped the idea to my boss, our VP and one other manager, but I kept the idea in my back pocket all this time, just waiting for the right time to drop it. Today was that day.

Because I had 7 months to plan out how to bring up this idea, I had determined the best way to convince everyone of how useful this would be. Sure enough, as I laid out my vision for usage, everyone in the room was convinced within minutes it was the right approach to take. It was something no one in the room had considered. Most were like me of 7 months ago, they didn’t even know that tools existed to do this type of analysis. The team immediately saw its value and started asking what other projects this might be used on!

What made this work, besides it being an idea whose time had come, was that I had done my research and the idea had time to grow in my mind. If someone had come to me 7 months ago and asked how we could better understand what is going on with our application, I would not have had a clue. Even 3-4 months ago, I would not have the full vision of what is now in my head. A year from now, after I’ve had time to tinker and play around with the ideas, it will be even better.

Its all about subconscious information processing. The idea of a robust analytics suite, not just for business metrics but for system and user metrics as well, was not created by me but it was one whose time has come for our organization. As I’ve been saying for months to anyone who will listen, “if we keep doing things the way we do them now, we’ll continue to fail in the same ways.” Its not that we fail a lot or fail badly, but everyone fails at some point. Keeping to the same processes only ensures that you’ll continue to fail in the same way.

So what ideas have been rolling around in your head? Drop them in the comments for us all to spend a little time performing a little subconscious information processing on them!

(BTW, I am considering writing up an eBook on my experiences with what I’m tentatively calling “Metrics-driven Analysis” so everyone can understand the vision I outlined to the team, along with details on how to determine what metrics to track. Anyone interested in this?)