Measurement and Appraisal in Counseling SC 609

 

Course Description

This course is designed for counselors, administrators, educators and consultants working within the educational environment to set and apply soundmeasurement principles in test design and interpretation for decision-making relative to educational programs. This course is an overview of basic principles of assessment in counseling, focusing on test and non-test appraisal instruments and the development of diagnostic skills. The course will include the selection, use of and interpretation of instruments appropriate for the appraisal of individuals and groups as related to the school environment. This course will offer information and techniques to the school counselor in making decisions using measurement of student performance and understanding the context in which educational programs are developed. Students will explore the concepts of measurement and evaluation, classroom test construction, creation and use of derived scores, selection and use of published measurement instruments, alternative assessment, and current issues.

Objectives-CACREP 2009 standards

1. Students will review the historical perspectives concerning the nature and meaning of assessment and describe current and future trends in educational measurement (CACREP A. 1, C.2. J.1.).
2. Students will recognize and identify statistical concepts, including scales of measurement, measures of central tendency, indices of variability, shapes and types of distributions, and correlations (CACREP I.1. I.3. I.4. J.1. J.2. J.3. ).
3. Students will recognize and identify basic concepts of standardized and non-standardized testing and other assessment techniques including norm-referenced and criterion-referenced assessments, environmental assessment, performance assessment, individual and group test and inventory methods, behavioral observations, and computer-managed and computer-assisted methods (CACREP C.2. C. 4. G.3. H.2. I.1. I.3. J.1. J.3. L.3. ).
4. Students will describe and examine the types and uses of validity and reliability and be able to differentiate between the two (CACREP C.2. C. 4. G.3. H.2. I.1. I.3. J.1. J.3. L.3.).
5. Students will recognize and discuss age, gender, sexual orientation, ethnicity, language, disability, culture, spirituality, and other factors related to the assessment and evaluation of individuals, groups, and specific populations (CACREP A.6. B.1. C.3. D.1. E.1. E.3. F.1. F.3. G.1. G.3. H.1. H.2. H.3. ).
6. Students will recognize and determine the appropriate process and procedures of testing including evaluating and selecting test, identifying the practical issue of testing, test administration guidelines and procedures and major issues and problems in test administration (CACREP G.1-4. H.1-5. ).
7. Students will recognize, identify and apply ethical and legal considerations in selecting, administering, and interpreting assessment and evaluation instruments including social, ethical, and legal issues and implementing professional standard and codes of ethics (CACREP B.1, E.1.-4. F.1-4. H.1.)
8. Students will operationalize an understanding of strategies for selecting, administering, and interpreting ability, aptitude, achievement, career and personality assessments and applying gathered data to techniques in counseling (CACREP Standard G. 3, H. 2. H.3. E.1.).
9. Students will demonstrate general principles and methods of clinical assessment, case conceptualization, assessment, and/or diagnoses of mental and emotional status for both the typical and diverse populations in a school setting.(CACREP G.2. G. 3. H. 1. H. 3. H.4.).
10. Students will understand and assess the theoretical backgrounds and influence of multiple factors that affect the personal, social, and academic functioning in the school environment. (CACREP G.1. H. 1. H. 5.).
11. Students will develop specific strategies and skills to analyze assessment information to evaluate the individual and assess the effectiveness of educational programs including making appropriate referrals within the school and/or community (CACREP G. 1-3.H.1-5.).

Time Requirements

This course will be offered over a 15-week period and is a 45 hour 3 credit graduate course. Modules will be completed one per week.

Hardware and Computer Skill Requirements

Students may use either a Macintosh computer or a PC with Windows 2000 or higher. Students should possess basic word processing skills and have Internet access as well as an active email account. Students also are expected to have a basic knowledge of how to use a Web browser, such as Internet Explorer, Mozilla Firefox, Safari, etc.

Curriculum Design

This course will emphasize the connection between the role of the counselor and appropriate student assessments. Students will be involved in a variety of tasks for completion of course requirements: readings, reports on readings, exams, direct application in the school setting, journal and reflection entries, projects related to real world learning environments, and development of instructional counseling strategies for specific content areas. Participants model appropriate assessment concepts to educational scenarios in each module. This is an online forty-five hour, three credit graduate level course completed over a fifteen-week period.

Course Materials

Required Text:

Assessment Procedures for Counselors and Helping Professionals ed 6^th by Rjobert J. Drummond and Karyn Dayle Jones.  (Students should also have access to the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition-Text Revision -DSM-IV-TR.)

Online readings and Web site reviews (including journal articles and best practices from the body of educational research) will be assigned during the course to enhance learning. These readings will be presented as annotated Web sites within the course content.

Student Performance Requirements:

Students will complete assigned readings, written assignments, applications/practice/direct observations in the educational setting and forum postings (as required) in each module. Students will complete tests and special projects as listed in the module content.

Session Outline

Module 1: Historical and Philosophical Foundations of Assessment.

Objectives:

1. Students will explain several uses and values of testing.
2. Students will explain the history of assessment and compare past history with current practices.
3. Students will assess current trends and issues in the practice of applying assessments by the school counselor in the educational setting.

 

Module 2: The Role of Statistical Concepts in Organizing and Interpreting Data

Objectives:

1. Students will describe the four scales of measurement.
2. Students will compare the measures of central tendency, and measures of variability of scores.
3. Students will assess ways to organize and present test data.
4. Students will compare ways to describe individual performance on two different tests.

 

Module 3: Developing a Clear Understanding of Test Scores: How to Score and Interpret Scores.

Objectives:

1. Students will explain the components necessary for scoring and interpreting tests.
2. Students will compare the different types of test scores.
3. Students will recommend appropriate frameworks to interpret test scores.

Module 4: Understanding and Applying the Concepts of Validity and Reliability.

Objectives:

1. Students will compare and contrast the meanings and the importance of validity and reliability of assessments.
2. Students will describe the differences between content, criterion-related and construct validity.
3. Students will apply evidence to judge the validity of a test.
4. Students will explain the meaning of reliability, errors of measurement and different types of reliability coefficients’.
5. Students will summarize the factors that influence the reliability of a test and describe different types of evidence to judge the reliability of a test.
6. Students will write a report of the types of validity and reliability presented in two different test manuals.

 

Module 5: Assessment Issues for Working with Diverse Populations

Objectives:

1. Students will identify and apply standards for testing diverse populations.
2. Students will apply and analyze major tests and assessment techniques to use with individuals from diverse populations.
3. Students will summarize the goals, objectives, issues and problems of working with diverse populations at school settings.
4. Students will develop skills to apply the techniques, assessment instruments and procedures necessary for use with ethnic groups in school settings.

Module 6: Understanding the Process and Procedures of Testing in the School Environment.

Objectives:

1. Students will identify difference sources for test information.
2. Students will understand the Decision Theory Model and apply it to making judgments and test selections for the school setting.
3. Students will analyze criteria to use in evaluating a test.
4. Students will describe and apply the standards for test administration.
5. Students will develop and use an observational instrument to use to evaluate procedures that an examiner should follow.
6. Students will research and write a report on the major issues in administering tests.

 

Module 7: Evaluating the Legal and Ethical Concerns and Issues in Testing.

Objectives:

1. Students will synthesize and apply the major codes of professional ethics into the understanding of test and testing practices.
2. Students will compare and contrast the different codes of professional ethics as they apply to evaluations.
3. Students will review and summarize the important federal and state legislation and court cases that affect the evaluation practices and processes.
4. Students will demonstrate knowledge of legal and ethical issues in evaluations and assessments.

 

Module 8: Issues and Overviews of Testing for Intelligence, Aptitude, Achievement, Career and Personality

Objectives:

1. Students will define the important terms for intelligence, aptitude, achievement, career and personality tests.
2. Students will define and analyze major theories for each test type.
3. Students will explore and determine what tests are appropriate to use in the school setting and why they are appropriate to use in the educational setting.
4. Students will explore the results of each type of test and describe how to report test results.
5. Students will research the factors to consider when interpreting test results to school-aged student, family, other educational professional or community agency.

 

Module 9: Problems and Issues of Clinical Assessment

Objectives:

1. Students will describe the models of clinical assessment.
2. Students will identify appropriate tests and describe effective techniques to apply these tests in the school setting.
3. Students will review ethical standard and legal issues of assessment in the school setting.
4. Students will explore issues and concerns for using of clinical assessment in the practice of school counseling.

.

Module 10: Developmental, environmental and other Assessment Issues in Education

Objectives:

1. Students will compare the general principles of development and the environment and describe how these principles of development and environment are exhibited in their current educational setting.
2. Students will compare and contrast the advantages and disadvantages of various assessments used to measure developmental and environmental factors.
3. Students will describe the issues of high-state testing programs.
4. Students will evaluate the different strategies to help individual school-aged students maximize test performance and locate sources available to use to improve test performance.

 

Module 11: Analyzing and Communicating Test Results

Objectives:

1. Students will identify important standards for AERA, NCME, ASCA and APA that apply to test selection and administration.
2. Students will develop skills necessary to accurately report test results in a written report.
3. Students will become familiar with different models of report writing.
4. Students will develop professional strategies to communicate results to students, teachers, parents, and other professionals.
5. Students will improve strategies for communicating test results with parents, students, and teachers.

Student Requirements

1. Students will complete weekly written assignments.                                  25% of grade
2. Students will complete weekly application to educational setting.           25% of grade.
3. Students will complete forum postings and responses.                             10% of grade
4. Students will complete special projects and or exams.                              40% of grade.

Grading:

100-93	A
92-85	B
84-77	C
76-69
69-

 

 

Proctored Exams:

General Information

Specific courses in each program require the completion of an online proctored exam using a Teacher Education University proctor via the ProctorU ^TM service. This serivce gives students the ability to take the proctored exam at a convenient time and location with the proctor viewing the student via Web cam.

 

Some Facts About the Exam

 

• Questions will reflect course content for modules 1-8.
• The content of the questions will reflect the type of assignments you completed in Modules 1-8, so it would be to your advantage to review your assignments prior to the proctored exam.
• This exam is designed to take approximately 2 hours to complete.
• Question types will include multiple choice, short answer, and essay.
• You will be able to log in to the exam the day after Module 8 assignments are due to your professor.
• Once you log in, the exam will only be available for 2 hours
• During the exam, you may use the course Web site for assistance, as well as other research sites such as Questia that may assist you in successfully completing the exam.

ProctorU^TM Technical Specifications and Instructions

The ProctorU^TM service requires equipment with the following specifications:

1. A high speed internet connection
2. A computer running Windows XP or later versions
3. A webcam with 640 x 480 pixel video resolution (provided by Teacher Education University upon student request)
4. Audio capabilities, either through the webcam or through a standalone microphone (most cams come with a built in mic)
5. Speakers must be connected to the computer and working
6. The ProctorU^TM program must be downloaded onto the computer (no installation necessary)

 

Each student will need to register for a free account with pupilcity at http://teu.pupilcity.com. 

Once an account is created, students should return to http://teu.pupilcity.com to schedule all exams at least seven full days before the desired exam time.  Students can change reservations until 24 hours before the exam, at which time they will need to contact pupilcity’s exam line at 205-870-8122. 

At the scheduled exam time, students should return to http://teu.pupilcity.com and follow the posted instructions to log into the assigned exam room.

Students should make sure their webcam and speakers are plugged in and turned on.  STUDENTS CANNOT TURN ON THE WEBCAM SOFTWARE BEFORE LOGGING INTO ProctorU^TM (this will create issues with the video feed).  Once in the room, students should follow the instructions to connect to the proctor and follow the verbal instructions to begin their exam.

 

Taking the Exam

The day after the Module 8 due date, course instructors will send students with the link to the exam.  Students will only be able to access the exam once.  They should not click on the link or attempt to open the Web page until they are ready to begin the exam.

Exam Rules

• The exam is designed to take approximately two hours to complete.
• Exams submitted more than two hours after they are initially accessed will not be accepted (proctors should carefully keep track of time spent on the exam).
• Students who do not propose a suitable exam site and possible proctor will not be allowed to take the exam.
• Students must not communicate with anyone regarding the exam, other than the proctor (or instructor or other designate), either verbally, electronically or otherwise, during the exam.
• The exam must be completed independently.
• Students must not discuss the curricular content of the exam with colleagues from class, either verbally, electronically, or otherwise, until after the final deadline for completion of the exam.
• If technical problems arise such that the student is unable to access the on-line exam, the proctor will notify the Teacher Education University staff, in order for the student to have an extended time period to complete the exam, if necessary.
• The proctored exam covers Modules 1-8.  It is not a closed book exam, students are free to use the textbook, browse the Internet, and the course Web site.  Because students will be tested on concepts, browsing absolutely does not help those who have not studied, but may be an aid to those who wish to occasionally clarify their thinking.
• Students and proctors may not remove a printed copy of the final exam from the exam site.
• If the exam is printed during the examination period, the copy must be destroyed by the student in site of the proctor before the student leaves the site of the final exam.
• It is recommended that students who may wish to browse during the exam open two copies of the browser during the exam (using one for browsing if necessary, and the other for the exam).
• It is recommended that students "save" frequently during the exam, or write down a few words to help remember which answer they have chosen for each question in the event that there is a power failure, or they accidentally close the browser window in which they are completing the final exam.

After The Exam

Upon completion of the exam, the student will log off from ProctorU^TM.  The instructor is notified that the exam is ready for grading.  Once the exam is graded, the grade will be posted in Maestro.

 

Student Academic Integrity

Participants guarantee that all academic class work is original. Any academic dishonesty or plagiarism (to take ideas, writings, etc. from another and offer them as one's own), is a violation of student academic behavior standards as outlined in the Teacher Education University catalog and is subject to academic disciplinary action.

References

Online Resources

Practical Assessment, Research and Evaluation 

http://pareonline.net/

Buros Institute of Mental Measurements

http://buros.unl.edu/buros/jsp/search.jsp

Educational Testing Service

http://www.ets.org/testcoll/index.html

Norm-Referenced Achievement Tests

http://www.fairtest.org/facts/nratests.html

Using Customized Standardized Tests

http://www.ericdigests.org/pre-9213/using.htm

Criterion- and Standards- Referenced Tests

http://www.fairtest.org/facts/csrtests.html

Instructional Goals and Objectives

http://www.personal.psu.edu/staff/b/x/bxb11/Objectives/

Creating Multiple-Choice Items

http://trc.ucdavis.edu/trc/services/testing/item.html

Using Item Analysis To Test The Test

http://www.fsu.edu/~ids/fac2002/ma_item_analysis.htm

Item Analysis

http://www.msu.edu/dept/soweb/itanhand.html

Improving Reliability and Validity

http://oct.sfsu.edu/assessment/evaluating/htmls/improve_rel_val.html

Stiggins, Rick, (2007). Assessment through the student's eyes. Educational Leadership, 67 (8), 22-26.

Stiggins, Rick, & Chappuis, Jan. (2006). What a difference a word makes: Assessment FOR learning rather than assessment OF learning helps students succeed

. Journal of Staff Development, 27(1), 10-14.

Stiggins, Rick & Chappuis, Jan. (2005). Using student-involved classroom assessment to close achievement gaps. Theory Into Practice 44(1), 11-18. Stiggins, Rick. (2005). From Formative Assessment to Assessment FOR
Learning: A Path to Success in Standards-Based Schools . Phi Delta. Counseling and Student Services Clearinghouse.

Other References:

American Counseling Association. (2003). ACA code of ethics. Alexandria, VA. Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). text revision. Washington, DC: Author.

American Psychological Association. (2002). Ethical principles of psychologists

and code of conduct. Washington, DC: Author.

Association for Assessment in Counseling, American Counseling Association. (2003).Responsibilities for users of standardized tests (3rd ed.). Alexandria, VA:

Arter, Judith. (2002). Rubrics, scoring guides, and performance criteria. In Understanding scoring rubrics; A guide for teachers, edited by Carol Boston. College Park, MD: ERIC Clearinghouse on Assessment and Evaluation.

Black, P.l, & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139-148.

Bennett, R. (2001). How the Internet will help large-scale assessment reinventitself. Education Policy Analysis Archives, 9(5).

Retrieved February 24, 2004 from the Education Policy Analysis

Archives Web site: http://epaa.asu.edu/epaa/v9n5.html

Chappuis, Jan. (2005). Helping students understand assessment. Educational Leadership, 63(3), 39-43.

Chappuis, Stephen. (2005). Is formative assessment losing its meaning. Education Week, 24(44), 38.

Fremer, J. & Wall, J. (2004). Why use tests and assessments:

Questions and answers. In J. Wall & G. Walz (Eds.), Measuring up: assessment issues for teachers, counselors and administrators (pp. 3–19). Greensboro, NC: ERIC Counseling

and Student Services Clearinghouse.

Hood, A. B & Johnson, R. (2002). Assessment in counseling. (3rd ed.). Alexandria, VA:American Counseling Association.

Joint Committee on Test Practices. (1999). Code of fair testing practices in education. Washington, DC: Author.

Sattler, J. M. (2001). Assessment of children: Cognitive application. (4 ed.). San Diego:

Sireci, S. (2004). Computerized adaptive testing: An introduction. In J. E. Wall & G. R. Walz (Eds.), Measuring up: Assessment issues for teachers, counselors and administrators (pp. 685–694).

Greensboro, NC: ERIC Counseling and Student Services Clearinghouse.

Kappan, Vol. 87, No. 04, December 2005, pp. 324-328.
Wall, J. (2004a). Harnessing the power of technology: Testing and assessment applications. In J. Wall & G. Walz (Eds.), Measuring up : Assessment issues for teachers, counselors and administrators (pp. 665–684). Greensboro, NC: ERIC.

Wall, J. (2004b). Internet resources in educational assessment: A webography. In J. Wall & G. Walz (Eds.), Measuring up: assessment issues for teachers, counselors and administrators. (pp. 757–770). Greensboro, NC: ERIC Counseling and Student Services Clearinghouse.

Wechsler, D. (1958). The Measurement of adult intelligence. Baltimore: Williams &Wilkins.