I worked as an electronics project engineer and project manager for Mattel Corp. in El Segundo, California from 1999 to 2001. During this period, I was the project manager for a girls' toy called Diva Starz.
To get an idea of what a Diva Starz doll is, please read this article from the Orange County Register.
Managing complexity: the Diva Starz project.
On my second day at Mattel, I was sent to San Luis Obispo to "save the Diva Starz project." While I was grateful for the show of confidence in my abilities, I would have preferred a bit more time to prepare...
The programming for the Diva Starz project had been contracted out. When I arrived, I found major problems, most of which were caused by poor communication and a lack of requirements. The basic problem was that Mattel's software engineering process was optimized for toys that speak one to four phrases in response to one or two switch closures. Squeeze the doll's left hand and it says one phrase. Squeeze the other hand and it says the other phrase. A toy of that complexity has requirements that can be explained in less than five minutes and remembered without documentation. The Diva Starz project was different. This toy had over 1600 phrases and a very complex play pattern; an experienced test engineer needed 40 hours to exercise all possible paths through the software. Instead of one in-house programmer writing the software in a day or two, a team of programmers was needed to write the software over a several month period. Compared to some of my previous projects at other employers, this was a fairly small software effort, but it was hundreds of times bigger than any software that Mattel had ever created. Mattel routinely spends three to six months attempting to get dozens of people to agree on the exact color of Barbie's hair clip or the exact wording of a recorded phrase. This is a very successful method of making Barbie dolls or Hot Wheels cars, but it is a disaster when applied to complex software. I needed to manage that complexity.
When I arrived at the software contractor's site, things were not going well. Mattel had dumped several random collections of recorded phrases on them with no hint as to what the play patterns were supposed to be. The problem was that with simple toys, the play pattern is obvious from listening to the recordings, and this methodology was blindly followed in an inappropriate situation. When the contractors asked what the dolls were supposed to do, marketing bypassed engineering and hired a "software consultant" Visual Basic experience only, no assembly language or even C) who spent a few hours talking to marketing then wrote a flowchart that literally made no sense. (There are better ways to define software requirements than flowcharts, but a good flowchart is far better than nothing.) The vendor spent a considerable amount of time trying to turn the flowchart into some sort of documentation that would specify the behavior of the software, but again and again ran into branches where it could not be determined what to do, paths with dead ends, etc. The good news was that the software modules that handled I/O, memory management, multitasking, etc. were extremely high quality. The contractor was doing everything possible to make the project a success and had the framework in place to do so; all they needed was a definition of the play patterns.
At this point the project was over budget, seriously late, and had no end in sight. The vendor kept asking Mattel what they wanted the toy to do, so Mattel sent a toy designer to San Luis Obispo to help. This designer was brilliant, with a clean vision of how to make a toy that was a huge hit. Alas, there were no programming skills to go with the toy design skills, and the resulting endless changes did nothing to help playability. The software engineers were some of the best that I have ever seen. They were well managed. The toy designer was a world-class expert at making toys that are fun. Despite all of these advantages, the project was a runaway that was heading for cancellation. It was my job to stop that from happening. What this project needed was a set of well-documented requirements.
I returned to Mattel and attempted to educate everyone concerned about how to manage an engineering project of this complexity. I started out by writing this memo, which started some valuable word-of-mouth lobbying in favor of my ideas, and I started collecting data that I could incorporate into the requirements document.
My next step was to build up my credibility as a project manager. I was new, with no track record of past success at Mattel. Mattel was seeing a contractor who kept missing deadlines. The contractor was seeing a customer that refused to provide promised information. Mattel insisted that the flowchart had all of the required information. I was running into brick walls in my efforts to define the requirements. I had to somehow convince the decision makers at Mattel that the flowchart they had provided was useless. The managers in engineering wouldn't listen to me because I had no track record, and the managers in the other groups relied on the managers in engineering to tell them whether I was right. I had to somehow show a bunch of people who had never seen a flowchart before how to evaluate the flowchart for themselves and understand that it was useless.
Here is how I solved the flowchart problem; I asked for help from marketing (the ones who provided the flowchart) to conduct a play session. One person would pretend to be a child playing with the toy and say things like "press yes button", "press no button", "put on green dress", etc. to emulate the various things that the microprocessor had inputs to sense. The other people in the meeting would pretend to be the doll, using the flowchart to decide what phrases to say, whether to move the head, etc. and thus emulate the various things that the microprocessor had outputs to control. I arranged to have the session videotaped so that I could send the tape to the vendor as a guide to the expected behavior. I made sure that everyone concerned understood that any spot where they could not figure out what the doll was supposed to do was also a spot where the programmers would not be able to figure out what to program the doll to do.
As I knew would happen, marketing found themselves unable to follow the flowchart that they were so sure was playable. They tried several entry points, but each time they became stuck with no clue as to what the doll's behavior should be in that situation. Suddenly everyone understood my concerns about the quality of the flowchart. The next day the request came down to engineering to create a new flowchart. Despite the task being assigned to two assembly language coders with no experience in complex software design, it was a big improvement. The flowchart went through several iterations, getting better and better, until it was decided that it was "done" by the simple method of refusing to playtest it any further. Still, it was a lot better - in fact, it was good enough to save the project. Using the new flowchart, I worked with the vendor to refine the requirements and get the software written.
My next goal was to restore the vendor's credibility by improving their ability to meet Mattel's needs. I was expecting that I would have to address various technical issues, but as I dug into the details, it became apparent to me that the contractor had done a good job on the technical details. In the area of project management, however, they were not prepared to deal with a large bureaucracy like Mattel. The first thing I did was to throw out the fictional "schedule" that everyone knew was impossible to keep. I replaced it with a promise that after two weeks I would provide the first draft of a real schedule based on real requirements. In order to help everyone involved to understand the importance of honest schedules, I distributed this chapter from a book Tom DeMarco. As I worked on the schedule, I replaced the fantasy with measurable milestones and negotiated the scope of the project and the resources assigned to it in order to get the toys in the stores by the Christmas selling season. This effort turned out rather well. After a couple of minor slips as we implemented the system, the contractor settled into a steady pattern of meeting every milestone on time and under budget.
Next, I addressed the quality of the program. With thousands of audio segments, a very complex play pattern and no requirements to follow, whether or not the toy was working properly was a matter of opinion and debate. As each software module was completed and shown to marketing, it became a source of endless change requests. I set up a procedure for requesting changes and reporting unwanted behavior. I also set up a two-person independent software-testing group and had Mattel lease an office a block away from the software developers to house them. The software testing was incorporated into the schedule; a module was marked as being complete when the independent testers confirmed that it met the written requirements, not when the software developers decided that it was complete. We started out with manual testing using checklists and started developing automated regression tests. Incoming change requests went through the independent testers, who bounced them back to Mattel if the problem could not be duplicated or was a duplicate of a previous request. The rest were sorted into a prioritized list so that low priority items would not eat the time needed to work on high priority items. The new software testing methods worked out quite well, and the developers continued meeting milestones. In addition, I encouraged the vendor to take responsibilities away from me and assign them to in-house staff, hiring new people where appropriate. This allowed them to be less dependent on me, and to apply the new methods to other customers.
In the end, we delivered the software, the doll went into production, and Diva Starz was Mattel's top selling new toy for the year 2000. This happened through word-of-mouth about how fun the toy was - marketing canceled all print and television advertising for Diva Starz because they didn't believe that the software would be completed on time and that the toy would not sell well even if we completed it. The same word-of-mouth has made Diva Starz a much better selling toy in the year 2001 than expected. The vendor is now capable of handling large corporate customers such as Mattel, and has had considerable success in the competitive toy design field.
I was also involved in solving some interesting hardware and manufacturability problems, but that effort was spearheaded by the engineer who designed the original hardware - I just assisted. I suspect that things would have gone smoother we had either let the vendor design the hardware and software together or kept the original hardware design, but by the time I was hired as project manager the hardware had been redesigned in China.
(Note: Mattel uses non-standard names for various engineering functions, and the name of my particular department changed several times while I was there. Rather than use the confusing department names, I have translated them into standard terms such as "Engineering" and "Marketing.")