People do come and go-that's life. Don't be disheartened if they go. Just treat it as a valuable opportunity, learn the lessons, and go back to Step 1 all over again!
Microsoft Certification and the Testing of Developers
Mark Pearce
Regardless of how much or how little they are paid, bad software developers are always extremely expensive. Hiring developers can all too often be a hit-or-miss affair. You talk to a guy, ask him some questions about his current and past work, and then ask him to answer a few ad-hoc technical questions. He seems personable, maybe even enthusiastic. Perhaps he stumbled over some of the technical questions, but he answered a couple of the others in convincing detail. He certainly seems to be able to "talk the talk," at least superficially. He knows all the latest buzzwords-COM+, MTS, ActiveX servers, polymorphism, etc. But if you don't have, for example, TMS's "Test from Hell," how do you know if the person that you're about to hire has constructed a facade of technical knowledge specifically to cover up his lack of real-world experience? And how do you measure him technically against the developer that you interviewed yesterday? Given the extremely large sums of money that can be commanded by developers nowadays, what would you give for some form of objective and comparable measurement of a candidate's job skills?
One approach is professional certification. Microsoft offers several different types of technical certification in their products, each designed to show a high level of competence in performing tasks related to the skill being tested. From our point of view, the most important qualifications are Microsoft Certified Professional (MCP) and Microsoft Certified Solution Developer (MCSD). The other popular qualification is Microsoft Certified Systems Engineer (MCSE).
An MCP has passed a mandatory product exam, for our purposes either Visual Basic 4 or preferably Visual Basic 5 (exams #70-065 and #70-165, respectively). Note that these are due to be replaced in the latter half of 1998 with the new Visual Basic 6 exam. The MCSD certification is more advanced and aimed at developers who have to design and develop custom business solutions with Microsoft development tools and technologies. It consists of two mandatory core exams, Microsoft Windows Architecture I and II (exams #70-160 and #70-161), and two elective exams from a list that includes Microsoft Visual Basic 5 (#70-165), Microsoft Access for Windows 95 (#70-069), and Implementing a Database Design on Microsoft SQL Server 6.5 or 7.0 (#70-027 or #70-029). Finally, the MCSE qualification is aimed mainly at people who implement and support networked information systems with Microsoft NT and Microsoft BackOffice. It consists of four mandatory core operating system exams and two elective language/product exams.
Most of these exams cost $100 each to take, are closed-book and computer-administered, between 60 and 90 minutes in length, and consist of a series of multiple-choice questions. They are designed to test the specific skills needed for the job functions being tested. Obviously every job function requires different levels of skills, from memorizing facts to analyzing scenarios, designing solutions and evaluating options. The exams aim to measure a candidate's performance in these job functions rather than just a candidate's knowledge of a product or terminology. In my experience, the questions are, on the whole, well researched and well written and do indeed perform a reasonable job of measuring a candidate's specific job skills. Microsoft insists that none of the exams contain "trick" questions. So let's just say that occasionally a candidate has to read a question very carefully and probably several times-a few questions are more about reading comprehension than about knowledge of the product or skill.
While the exams do have some professional standing in the IT industry and are considered to be fairly demanding, they also have a couple of inherent drawbacks. The first is that each exam is designed to measure general competence in a particular area. Often more useful is the measurement of a specific competence, such as the use of DCOM or an understanding of how to implement ActiveX servers efficiently. As Visual Basic becomes larger and more complex, there is a need for developers who specialize in specific parts of the product. If you're building small teams, with each team specializing in one area, construct your own tests (verbal, written, or computer-administered) to measure thorough competence in that area. By all means follow the Microsoft test format if you want, but engineer your questions to concentrate on the specific areas you need. Refine the questions over time based on candidates' results and feedback. Eventually you will have a test suite tailored to your own requirements.
The second drawback is that the Microsoft exams all use a multiple-choice format. This enables computer grading of tests, without the need for human involvement. However, it is sometimes useful to present open-ended questions, such as asking a candidate to write Visual Basic code to solve a particular problem. Although these types of questions are more subjective and require expert human input to judge the answers, they can also give you more insights into a developer's thought processes and way of working than those questions that use the multiple-choice format. You can also ask more unusual questions. One of my favorites is to ask a candidate what question he or she would add to the test, and how would he or she answer their own question.
My advice is to use the Microsoft exams as a useful tool, and then supplement them with a test of your own design to refine your knowledge of a candidate's technical competence. Be prepared to spend some time constructing your test to ask the right questions in a non-ambiguous manner. You are likely to be paying any new hire tens of thousands of dollars, or even hundreds of thousands. It makes no sense to hire just on gut feelings or on inadequate testing.
