Education has evolved to become an important sector in the development space. In countries such as India, imparting free and quality education is a huge challenge. ASER reports have always highlighted the dark reality of education in India. The poor status of learning outcomes of children has led to the entry of a vast number of social organizations and huge disbursement of CSR funding into education. With the growing maturity of these organisations, people have started to realize that intervention alone cannot be termed as impact and hence the need to conduct separate and independent evaluation techniques have become indispensable in the development domain. Employing diagnostic tool to identify learning levels of students at the base and end of the intervention is a popular tool used by education-related organisations to evaluate their impact. The article aims to provide an introduction to diagnostic assessment and steps to create a math diagnostic assessment to identify the learning levels of students.
7 steps to create Diagnostic Math Assessment
Step 1: Resource Check
To create an assessment, the following resources are required to be in place:
List of learning outcomes derived from the curriculum
Reference Textbook, eg. Balbharati
Other resources of assessment, eg. Saksham, PSM etc.
Step 2: Creating a blueprint of available assessment
Blueprint is the structure of any assessment. Blueprint includes topic from the curriculum, learning outcome that the question aims to evaluate, sample question and score/marks for the question.
Map out the available validated assessment, if any. Breakdown it into the area, topic, type of question and score.
Convert score into percentage or weightage. Blueprint is ready for that source.
The purpose to create a blueprint for another source is to get an idea of how the weights are generally distributed among the topics. It helps to decide weightage for our assessment tool.
Step 3: Create a fresh blueprint
Go to the curriculum (refer MSCERT website and download from syllabus)
Download the curriculum for the required subject.
Identify broad areas of the curriculum. Breakdown areas into topics/subtopics.
Create an excel sheet for all areas/topics across all required grades.
Identify important areas and topics based on the textbook, learning outcomes and experience. The logic is more important is the topic, higher is its weightage
This is a critical step which requires a lot of professional judgement. Now create a blueprint based on areas and topics identified.
Put formula for percentage to let score comes automatically. Giving percentage to topics will depend on the weightage identified in the 5th point.
Step 4: Create questions
From the blueprint created in step 3, start creating questions aligned to mapped out learning outcomes.
Based on the area and weightage identified, design type of questions to be framed.
Learning outcome needs to be broken down to specific skills.
Keep assessment scaffolded.
*Scaffolding needs professional judgement.
Note: Assessments are always made keeping objective of assessment in perspective. If the objective is to do a baseline, lower learning levels are generally tested.
Step 5: Create Answer Key
Once the questions for assessment are ready, parallelly create an answer key as well as the data entry structure for the assessment. This helps the investigator to track the scores and evaluate the areas in which students are struggling.
Step 6: Review of the assessment
Make sure to review the assessment as well as the answer key. It is important to check assessment for spelling, grammar, (Math) language, names and images making sure that they are not plagiarized or sensitive. The third-party check is to ensure there is no bias in the assessment.
Step 7: Piloting the assessment
Once a diagnostic math assessment is created, conduct a pilot with students of specific grade for whom the assessments are designed. Pilot studies are conducted to check for the efficiency of the tool, time taken by students to complete the assessment and to know common understanding and interpretation of questions by the students. Depending on the learning from the pilot, make necessary changes to the tool. A pilot is successful if you are able to get a positive response and not major changes are required to be done to the tool. Once the pilot is successful, your diagnostic math assessment tool is ready to be implemented for impact evaluation.
Conclusion
While you have made a baseline assessment, a good practice is to make end-line parallel to the baseline. Endline assessment should be aligned to the baseline, meaning at least 80% of the same learning outcomes are tested with 20% of the variation. This is done to compare progress from both baseline and end-line assessment. Data can be analysed by identifying the learning levels of students as well as identifying competencies depending on the objective of the program. At last, diagnostic assessment is a great way to identify a student's learning level and give customized support in the classroom.
Important terminology
Curriculum: Curriculum is the universe that assessment will be limited to. Any assessment cannot go out of the curriculum. A curriculum is defined by the educational boards such as NCERT, MSCERT etc. For Maharashtra board, assessments should be restricted to curriculum defined by MSCERT for the specific subject. The curriculum is available on MSCERT website.
Learning outcomes: Learning outcomes are derived from the curriculum. Learning outcomes can be defined as a short version of the curriculum. All teaching pedagogy should be aligned to defined learning outcomes. Learning outcomes are generally available in initial pages of Balbharati textbooks.
Reference book: Reference books such as Balbharati & NCERT is the medium to achieve learning outcomes. It may or may not cover all aspects of the curriculum. Therefore it is expected to teach from different textbooks.
Syllabus: Syllabus is the summary of topics to be covered. Different textbooks provide different syllabus aligned with the curriculum.
Diagnostic assessment: A diagnostic assessment is a tool to measure individual student’s learning skills and help educators to clearly identify the areas where students need help with understanding a concept.
Impact Evaluation: Evaluation is an independent and rigorous assessment of either completed or ongoing project activities to determine the extent to which the planned objectives are being achieved. While monitoring provides real-time information, evaluation provides an in-depth analysis of the impact.
Baseline/Endline Assessment: Baseline assessments are conducted before the intervention starts. It is done to know the level of students at learning levels. Endline is conducted at the end of the year. Baseline and Endline assessments are compared to evaluate the impact.
The author Pranjali Hardikar is an Associate with the Research, Monitoring & Evaluation vertical Leadership For Equity. Pranjali holds a Masters degree in economics from the Symbiosis School of Economics, Pune and has previously worked with Teach For India and National Bank for Agriculture and Rural Development (NABARD). She can be reached through email at pranjali.h@leadershipforequity.org.