(ISC)2 Conducting CISSP Exam Survey
Last month I reported a friend's experiences with the CISSP exam. This week I received an email from (ISC)2 regarding a survey of the CISSP exam. It reads in part:
"(ISC)2 would like to extend to you the opportunity to provide key input into the content of the CISSP® examination. With assistance from Schroeder Measurement Technologies, Inc., (ISC)2’s services entity,(ISC)2 is conducting a CISSP job analysis study through an online survey. The purpose of the job analysis study is to ensure the currency of future CISSP examinations.
As a CISSP certificate holder, we are asking you to participate in the survey. *Your responses are valued and essential*. We ask that you set aside 20 to 30 minutes of your time no later than Thursday, July 14, 2005 to complete the online survey."
Once I started taking the survey, I saw these guidelines.
"A comprehensive list of important job tasks performed by an Information Systems Security Professional is presented on the following pages. Please provide your ratings to the tasks in relation to the practice of Information Systems Security Professionals at your work site."
I was initially excited by the prospect of ISC2 using survey results to revamp the terrible CISSP exam... until I started looking at the survey. Here are a few screen captures. To the right of each item are radio buttons saying "Not Performed, Of No Importance, Of Little Importance, Moderately Important, Very Important, Extremely Important."
This first section presumably asks if these technologies are important. Is this the way an exam should be written? The next screen shot is even worse.
What am I supposed to do here, say a Value Added Network (VAN?) is "Moderately Important" while a hub is "Of Little Importance"?
I looked at one more section, shown below, before giving up.
This survey is a disaster. The CISSP certification should be about security principles. ISC2 should take a look at a wonderful book like Ross Anderson's Security Engineering to figure out what matters. Asking me about hubs or CHAP or the PSTN is foolish. Whatever results ISC2 thinks it gets from this survey will not improve the certification. Again, the only value CISSP retains is its Code of Ethics.
"(ISC)2 would like to extend to you the opportunity to provide key input into the content of the CISSP® examination. With assistance from Schroeder Measurement Technologies, Inc., (ISC)2’s services entity,(ISC)2 is conducting a CISSP job analysis study through an online survey. The purpose of the job analysis study is to ensure the currency of future CISSP examinations.
As a CISSP certificate holder, we are asking you to participate in the survey. *Your responses are valued and essential*. We ask that you set aside 20 to 30 minutes of your time no later than Thursday, July 14, 2005 to complete the online survey."
Once I started taking the survey, I saw these guidelines.
"A comprehensive list of important job tasks performed by an Information Systems Security Professional is presented on the following pages. Please provide your ratings to the tasks in relation to the practice of Information Systems Security Professionals at your work site."
I was initially excited by the prospect of ISC2 using survey results to revamp the terrible CISSP exam... until I started looking at the survey. Here are a few screen captures. To the right of each item are radio buttons saying "Not Performed, Of No Importance, Of Little Importance, Moderately Important, Very Important, Extremely Important."
This first section presumably asks if these technologies are important. Is this the way an exam should be written? The next screen shot is even worse.
What am I supposed to do here, say a Value Added Network (VAN?) is "Moderately Important" while a hub is "Of Little Importance"?
I looked at one more section, shown below, before giving up.
This survey is a disaster. The CISSP certification should be about security principles. ISC2 should take a look at a wonderful book like Ross Anderson's Security Engineering to figure out what matters. Asking me about hubs or CHAP or the PSTN is foolish. Whatever results ISC2 thinks it gets from this survey will not improve the certification. Again, the only value CISSP retains is its Code of Ethics.
Comments
You ever listen in on the CISSP mailing list? From the rumblings there and the limited response from ISC^2 management, it looks like things are probably going to get worse before they get better. The Powers that Be at ISC^2 just don't get it. And I don't know why. I can only guess that they've become more concerned with making money than they have with supporting the 'Gold Standard' of security.
Martin
This is a very unreliable way of getting crucial input and it really makes me wonder whether the people building a test from this survey are qualified to build any tests at all - apparently stats 101 or intro psychology aren't useful to a CISSP. ;)
mjr.
Day by day I'm running away from this CISSP thing...
-Ronaldo
When the training side of your business has taken over the management side, it is too late.
Stop fighting in the training market and start doing your job in the psychometric market. Start listening to your constituents because what they are telling you today is what will kill you in the future if you do not listen and act now.
It is about time the old boys club that calls themselves "The Board" gets dismantled and some people who really care about the value of the certifications gets in place.
This is not about to change considering that nomination for the board has to be approved by the board.
Mr. Disappointed
By the way, if you really believe that an Infosec Pro doesn't need to know what's CHAP, to understand the security implications behind this particular protocol and how this influences the security that it provides, I think you're really mistaken. That's not the only sort of thing that the professional must know, but obviously is something important.
If they would be transparent, they would publicly open their CBK and they would not distribute what they call a study guide????
This study guide of theirs is totally inadequate. It is a series of high level bullets that does not help a potential candidate to the exam.
I invite your replies as well. khenry@isc2.org
Your idea of how to construct a useful survey is "completely misunderstood." Your goal may be noble but your implementation will not yield the results you want.
Anderson Ramos,
I know full well what is involved with job analysis. I am part of the BSD Certification Group which published a Task Analysis Survey and the results of that survey.
You should "get informed before publishing things" on my Web site.
$500 US for a dinky scantron test. thank's for the memories.