Planning a Pilot

Educator or District Leader analyzes needs, develops goals, and creates a system of inquiry to effectively implement a pilot.
Made by Digital Promise Marketplace and Research
APPLY

About this Micro-credential

Key Method

The educator or district leader critically examines program implementation and outcomes to make informed, evidence-based decisions about programs and products. The first step of this process is successfully planning a pilot and submitting a proposal.

Method Components

There are multiple steps involved in planning a pilot. Each step should be completed prior to initiating a pilot to ensure a successful pilot study.


1. Identify the school or district’s need and provide a rationale (including evidence) to support the existence of this need.

In order to plan a pilot, educator and district leaders must first identify a need within their context. The need should be based on evidence, gathered through assessment scores, interviews or focus groups, surveys, or other data measures related to student or educator success and experience (i.e., a specific student group has the lowest standardized test scores in a particular subject, therefore there is a need to find an edtech tool that better supports and teaches this student population in the particular academic subject).


2. Explicitly define the pilot goal.

The goal should be your vision of the need being met, meaning the ideal outcome for the solution being piloted. Additionally, explain how you will or how you have already included stakeholders (e.g., educators) throughout the goal defining process in order to gain buy-in. Be as explicit with the goal as possible to ensure all stakeholders understand the purpose of the pilot and what the efficacy of the solution will be measured against.


3. Articulate a research question to direct the pilot study (optional).

The level of evidence required to assess the efficacy of an edtech tool may vary depending on the type of product or program and the number of students involved. In cases where significant evidence is required to make a purchasing decision, structuring the pilot with specific research questions can be helpful.


strong>4. List the metrics and the methods to collect the data to understand the efficacy of the piloted solution.

This step in planning a pilot requires the identification of metrics necessary to evaluate the effectiveness of the piloted tool along with the collection and design of data collection methods. With stakeholders (district leaders and, when appropriate, educators), determine the metrics necessary to determine whether the goal has been met or the data needed to answer the research question. Once these data are identified, determine the data collection methods that will be used to gather that data (i.e., pre-post assessment scores, educator interviews/focus groups, student focus groups, student pre-post surveys, educator pre-post surveys, etc.). If the data collection methods require either finding or designing instruments, include the source you have identified that provides the data collection instrument(s) or include the designed instrument in your submission (including citations when references occurred).


5. Determine a professional development or teacher training plan for the solution.

Discuss with the vendor whether PD is offered, the frequency and duration of the PD, and how educators can expect to gain ongoing support. Additionally, discuss with district leaders how often the district will provide PD opportunities or training sessions on the solution. Describe the educator’s opportunities in your submission and how these opportunities were determined.

Research & Resources

Supporting Research

Resources

Submission Requirements

Submission Guidelines & Evaluation Criteria

To earn the micro-credential, you must receive a passing evaluation for Parts 1 and 3 and a ‘Yes’ for each component in Part 2.

Part 1. Overview Questions

(300 word-limit)

Identify a need within your district and plan a pilot to determine whether an edtech tool serves as a solution to meeting this need. This need can involve students’ assessment scores, engagement levels, social-emotional growth, academic growth in a specific construct, and so on. Please complete Part 1 before engaging with Part 2.

Describe the pilot plan, making sure to include:

  • A description of the need
  • How the need impacts students or the school or why this need should be addressed
  • The proportion of students impacted by this need within the school or district that will pilot the edtech tool as a solution
  • Previous attempts to meet this need, if applicable

Part 2. Work Examples / Artifacts

To earn this micro-credential, please submit a document with the following pilot process plan:

  • What is your identified district or school need? How did you identify this need?

Ex: Through disaggregation of assessment scores, district leaders realized that English learners are not receiving the educational support they need to pass grade level tests at the average district rates.

  • What is the goal for the solution to be piloted? What does “meeting the need” look like?

Ex: Our district will pilot an edtech tool that will support English learners in reading comprehension to increase grade-level reading skills among the population.

  • What is the pilot’s research question? (Optional)

Ex: To what extent does the edtech tool improve student reading comprehension?

  • What data are you going to collect to determine whether the goal has been met? What instruments will be used to collect the data? Please include the instruments in your submission, or URLs to the instruments you will use from other sources.

Ex: In order to assess the product’s efficacy, we will conduct a mixed methods study. We will use quantitative data from the standardized district-wide reading comprehension assessment, where we will pull Lexile levels from pre and post tests to measure change. We will also conduct pre and post educator focus groups and administer educator and student pre-post surveys to understand levels of engagement and applicability to state standards and district curriculum, and to assess changes in attitudes and skills. We have written the interview protocol for teachers and provided the form below, as well as the pre-post surveys.

  • What professional development (PD) will be provided to educators involved in the pilot? Will the district provide PD? Will the vendor provide PD? Will PD be online, in-person, or both? Will there be ongoing support? If so, how will educators access the support?

Ex: The vendor will provide in-person PD to educators a month before the pilot begins and half-way through the year. The vendor will provide ongoing support through a single point-person who all educators can contact (via email, phone) directly whenever they need assistance. The district will provide online PD two times throughout the year, as well, and ensure that the technology coordinator visits each classroom twice in the first semester and once in the second semester.

Part 3. Reflection

(200-word limit)

After you designed the pilot plan using this protocol, write a reflection that addresses the following:

  • How did this change the way you typically identify a problem you intend to address? If it did not change the way you identify a problem, please describe the ways in which this strengthened your existing process.
  • Did this impact the way you intend to determine whether an edtech tool is being used successfully? Why or why not?
  • Did this change the way the district/educator leaders involve other educators? Did educators report feeling more or less involved?
  • What lessons can you take away from this PD experience to continue to use in future pilot planning processes and pilot processes?

  • Except where otherwise noted, this work is licensed under:
    Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
    http://creativecommons.org/licenses/by-nc-nd/4.0/

Requirements

Download to access the requirements and scoring guide for this micro-credential.
Requirements for Planning a Pilot
How to prepare for and earn this micro-credential - in a downloadable PDF document

Ready to get started?

APPLY