BY: Collin Quiring

This is part three on Microsoft Certifications.  Part one and two dealt with the question of “Is it worth it” from the perspective of an employer and an individual.  In writing those, I became interested in the process of how a test comes into existence.   That is what this article is about.

As Microsoft technology is replaced by newer versions, so are the tests.  If I were to tell you that I am certified in Microsoft Project, you might be impressed; until I told you that it was Microsoft Project 1994.  Hopefully, you will be more impressed that I am certified in Microsoft Project 2007 and Microsoft Project Server 2007.  Keeping current on the tests is important.  But, how do the new tests get created?

According to Microsoft, tests now normally go through this development process:

To ensure the validity, reliability, and relevance of Microsoft Certification exams, developers create exams in eight phases:

1.       Job analysis: Exam developers break down the tasks performed within a specific job function, and identify the relevant knowledge, skills, and abilities needed when using a specific technology.

2.       Objective domain definition: Tasks are translated into a comprehensive set of more specific and measurable skills and abilities. The resulting list of objectives, or the objective domain, forms the basis for the development of certification.

3.       Blueprint survey: The objective domain is transformed into a blueprint survey, in which technical and job function experts rate the importance of each objective. This helps to determine the appropriate number and types of items to include on the exam. Contributors may be Microsoft Certified Professionals (MCPs) or MCP candidates recruited from within Microsoft or through online forums. Based on contributor input, exam developers prioritize and weigh the objectives.

4.       Item development: Exam developers write the exam items according to the prioritized objectives. Developers review and revise items to ensure that they are:

Technically accurate

Clear, unambiguous, and relevant

Not biased toward any population, subgroup, or culture

Not misleading or tricky

Testing for useful knowledge rather than obscure or trivial facts

Items that meet these criteria are included in the alpha item pool.

5.       Alpha review and item revision: A panel of experts reviews each item for technical accuracy. After the items are approved, they undergo a legal review.

6.       Beta exam: The reviewed items are beta-tested. During the beta exam, candidates comment on items. The beta exam allows Microsoft to evaluate the quality of the item in an actual exam situation, and helps ensure that only the best content is included in the live exam.

7.       Item selection and cut-score setting: The results of the beta exam are analyzed to determine which items should be included in the live exam. This analysis focuses on many factors, including item difficulty and reliability. Microsoft works with a panel of experts to review the technical accuracy of questions and to determine the final item pool for the live exam. The panel determines the cut score (minimum passing score) for the exam. This score differs from exam to exam, because it is based on the difficulty of the item pool and the expected performance of the minimally qualified candidate.

8.       Live exam: Prometric, an independent testing company, administers the final certification exam. The exam is available at testing centers worldwide.

I used to wonder why it took so long for the tests to come out in some areas after the software was released.  But, if you realize that the software sometimes isn’t finalized until just before it is being released, that means that test questions can’t even be created yet!

Read Parts One and Two here.