The educator or district leader critically examines program implementation and outcomes to make informed, evidence-based decisions about programs and products. The first step of this process is successfully planning a pilot and submitting a proposal.
There are multiple steps involved in planning a pilot. Each step should be completed prior to initiating a pilot to ensure a successful pilot study.
In order to plan a pilot, educator and district leaders must first identify a need within their context. The need should be based on evidence, gathered through assessment scores, interviews or focus groups, surveys, or other data measures related to student or educator success and experience (i.e., a specific student group has the lowest standardized test scores in a particular subject, therefore there is a need to find an edtech tool that better supports and teaches this student population in the particular academic subject).
The goal should be your vision of the need being met, meaning the ideal outcome for the solution being piloted. Additionally, explain how you will or how you have already included stakeholders (e.g., educators) throughout the goal defining process in order to gain buy-in. Be as explicit with the goal as possible to ensure all stakeholders understand the purpose of the pilot and what the efficacy of the solution will be measured against.
The level of evidence required to assess the efficacy of an edtech tool may vary depending on the type of product or program and the number of students involved. In cases where significant evidence is required to make a purchasing decision, structuring the pilot with specific research questions can be helpful.
This step in planning a pilot requires the identification of metrics necessary to evaluate the effectiveness of the piloted tool along with the collection and design of data collection methods. With stakeholders (district leaders and, when appropriate, educators), determine the metrics necessary to determine whether the goal has been met or the data needed to answer the research question. Once these data are identified, determine the data collection methods that will be used to gather that data (i.e., pre-post assessment scores, educator interviews/focus groups, student focus groups, student pre-post surveys, educator pre-post surveys, etc.). If the data collection methods require either finding or designing instruments, include the source you have identified that provides the data collection instrument(s) or include the designed instrument in your submission (including citations when references occurred).
Discuss with the vendor whether PD is offered, the frequency and duration of the PD, and how educators can expect to gain ongoing support. Additionally, discuss with district leaders how often the district will provide PD opportunities or training sessions on the solution. Describe the educator’s opportunities in your submission and how these opportunities were determined.
To earn the micro-credential, you must receive a passing evaluation for Parts 1 and 3 and a ‘Yes’ for each component in Part 2.
Identify a need within your district and plan a pilot to determine whether an edtech tool serves as a solution to meeting this need. This need can involve students’ assessment scores, engagement levels, social-emotional growth, academic growth in a specific construct, and so on. Please complete Part 1 before engaging with Part 2.
Describe the pilot plan, making sure to include:
To earn this micro-credential, please submit a document with the following pilot process plan:
Ex: Through disaggregation of assessment scores, district leaders realized that English learners are not receiving the educational support they need to pass grade level tests at the average district rates.
Ex: Our district will pilot an edtech tool that will support English learners in reading comprehension to increase grade-level reading skills among the population.
Ex: To what extent does the edtech tool improve student reading comprehension?
Ex: In order to assess the product’s efficacy, we will conduct a mixed methods study. We will use quantitative data from the standardized district-wide reading comprehension assessment, where we will pull Lexile levels from pre and post tests to measure change. We will also conduct pre and post educator focus groups and administer educator and student pre-post surveys to understand levels of engagement and applicability to state standards and district curriculum, and to assess changes in attitudes and skills. We have written the interview protocol for teachers and provided the form below, as well as the pre-post surveys.
Ex: The vendor will provide in-person PD to educators a month before the pilot begins and half-way through the year. The vendor will provide ongoing support through a single point-person who all educators can contact (via email, phone) directly whenever they need assistance. The district will provide online PD two times throughout the year, as well, and ensure that the technology coordinator visits each classroom twice in the first semester and once in the second semester.
After you designed the pilot plan using this protocol, write a reflection that addresses the following:
Except where otherwise noted, this work is licensed under:
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)