High Impact Learning Analytics (HILA) - Leadership Academy

After generating a report in the High Impact Learning Dashboard, Completed Courses dashboard, or Current Courses dashboard, you can download CSV files containing subsets and/or all of the data comprising the metrics. 

Download a CSV file containing all data used in the report

1. Run your desired report.
2. Once the report is generated, click on the Enrollments figure at the top of the report.
   
3. Once clicked, a table will appear. Click on the Download Data button at the top of the table and the CSV will be downloaded.

Download a CSV file containing a subset of data

If you want to download a subset of the data - for example, a list of learners labeled as Collaborators - you can download this data within the dashboard as well.

1. Within the dashboard, click on the specific item you wish to download data from. For example, in the screenshot below, the desired data is Collaborators. Note: Almost every figure and chart is clickable in the dashboards.
   
2. Once clicked, a table will appear containing the respective data. Click on the Download Data button at the top of the table and the CSV will be downloaded containing only the data chosen.

After generating a report in the High Impact Learning Dashboard, Completed Courses dashboard, or Current Courses dashboard, you can download a PDF of the report from within the dashboard.

1. Run your desired report.

2. At the top of the dashboard, click on the report tools icon, located near the Run button.

3. Click Download PDF and the file will download. Note: The PDF includes all of the charts and figures displayed in the dashboard, but will not display the learner data. Learn more about how to download CSV files in the HILD, Completed Courses, and Current Courses Dashboards.

This article outlines how you can run a High Impact Learning Analytics (HILA) report using the High Impact Learning Dashboard (HILD).

By generating a HILA report, you can view the analytics and insights from one or more cohorts. Reports can be generated for courses in progress and for completed courses.

• Learn more about our HILA measurements
• Learn how to download an HILA report PDF
• Learn how to download an HILA report CSV containing all of the data included in the HILA report

To run a HILA report, follow these steps:

1. Navigate to the HILD

2. Click on Filters at the top of the page.

3. Next, from the dropdown, choose the course you would like to report on. (Note: You can only run a report for one course at a time)

4. After choosing the course, select the cohort(s) you want to include in the report by clicking in the open text box and selecting each cohort to include.

Tip: If you have many cohorts to include, you can change the dropdown to “excludes” and then select only the cohort(s) you would like to exclude from the report. The report will then include all other cohorts that took the course listed in the Course section.

5. Finally, select the dimension you would like the results filtered by from the options listed. For help interpreting the output from this selection, learn more about our HILD measurements as well as the importance of properly setting up your learner data fields.

6. Click Run to generate your report. 

Tip: Almost every figure and chart is clickable in the HILD, allowing you to dive deeper into the data and gain valuable insights. For example, if you’d like to view the list of NPS promoters and the comments they provided, you can click the green portion of the NPS graph; if you’d like to view who was silent in the discussion activities in the course, you can click the light blue portion of the Discussion graph.

This article provides the data ranges for the Commitment metric in the HILA report.

• Learn how to generate a report using the High Impact Learning Dashboard.
• Learn about the data category ranges within the Engagement metric.
• Learn about the data category ranges within the Advocacy metric.
• Learn about the data category ranges within the Understanding metric.

The Commitment metric measures whether learners have agreed to take action or implement the ideas learned in the course. This measure combines (1) whether they said they were going to do so with (2) an algorithm that interprets how they said they would do so.

Commitment category ranges

• Very High: 80-100
• High: 60-79
• Moderate: 40-59
• Low: 20-39
• Very Low: 0-19

Our platform incorporates artificial intelligence to provide insights about how leaders are developing, what they believe, and the impact they are making. It helps learning leaders understand the hearts and minds of participants through analysis of dialogue from activities, interactions, and learning communities. As a result, organizations can better predict behavioral change, provide actionable insights, and improve decision making.

We collect individual and cohort performance data to identify top performers, learning advocates, and identify potential barriers for the initiative. While most virtual learning platforms use completion rates and surveys to measure impact, we aggregate data from multiple inputs and precise metrics to enable you to measure the impact of your leadership development programs.

Our analytics help you answer these questions

• How prepared are our leaders?
• How effective are we at developing our leaders?
• Are leaders acting differently?
• Are we seeing a positive outcome?

Metrics used to measure cohort programs in the High Impact Learning Dashboard

Engagement

Are They Actively Participating?

Participants who actively engage in the learning experience are more likely to understand and apply the information after the course has concluded. As a standalone metric, completion rate does not accurately reflect engagement. The engagement metric factors in participant completion rate, the volume and frequency of discussion posts, as well as quality and impact of those discussion interactions.

Advocacy

Will They Champion the Initiative?

To create meaningful change within the company, the leadership team must be champions of the cause and lead by example. The Advocacy metric measures participant “buy-in” and shows which leaders have become true believers who are willing to set aside past practices so they can apply and master these new skills.

Understanding

Do They Comprehend the Information?

In order to drive and implement change, your leaders must be able to explain and discuss the ideas, concepts and practices presented in the course material. The understanding metric uses data collected through Natural Language Processing (NLP) algorithms to compare words and phrases shared in discussions against the content in each course activity, yielding a reliable and validated estimate of leaders’ comprehension.

Barriers

What is Preventing Application?

Potential barriers can be identified which predict how organizational structures, systems, culture and priorities might prevent the successful implementation of new practices. The information is invaluable in creating leadership teams that can truly work as a team to implement an idea, preempting and solving problems before they derail critical initiatives.

Commitment to Action

Do We Have the Resolve to Make this Change?

Encouraging leaders to make a commitment to the initiative is the most effective way to solidify behavior change. Commitments work because people value consistency between what leaders say and what they do. Research shows that written commitments are more effective than verbal commitments and those made publicly are more effective than commitments kept private, creating a dual benefit to this metric.

Results by Dimension

Using the Results by Dimension Selector in the filters section of the HILD, you can view several metrics filtered by a chosen data field. For example, if you select Region in the Results by Dimension Selector, you can view the average Engagement, Advocacy, and other metrics for learners from each Region. Note: If you do not have consistent and accurate data, the codes in this section may not appear as you would like. Learn more about the importance of consistent and accurate data.

To learn more about our analytics and/or schedule a demo, please contact your Customer Success representative.

This article provides the data ranges for the Understanding metric in the HILA report.

• Learn how to generate a report using the High Impact Learning Dashboard.
• Learn about the data category ranges within the Engagement metric.
• Learn about the data category ranges within the Advocacy metric.
• Learn about the data category ranges within the Commitment metric.

The Understanding metric measures the degree to which a learner has comprehended the material in the course. It is measured using a combination of (1) how well the learner believes they could explain the material to someone else and (2) Natural Language Processing (NLP) algorithms comparing words and phrases shared in discussions against the content in each course activity, yielding a reliable and validated estimate of learners’ comprehension.

Understanding category ranges

• Very High: 70-100
• High: 65-69
• Moderate: 55-64
• Low: 25-54
• Very Low: 0-24

This article provides the data and associated ranges for the Advocacy metric in the HILA report.

• Learn how to generate a report using the High Impact Learning Dashboard.
• Learn about the data category ranges within the Engagement metric.
• Learn about the data category ranges within the Understanding metric.
• Learn about the data category ranges within the Commitment metric.

Advocacy measures whether learners would actively advocate for others to participate in the course, combining measures of (1) whether they would recommend the course to others with (2) whether they found the course valuable.

Advocacy category ranges

• Strong Advocate: 84-100
• Advocate: 70-83
• Weak Advocate: 56-69
• Neutral: 42-55
• Weak Critic: 28-41
• Critic: 14-27
• Strong Critic: 1-13

Value

The Value metric comes from the end of course learning measurement survey. Learners are asked the extent to which they agree that the course was valuable to them in their role, on a scale from Strongly Disagree to Strongly Agree.

Net Promoter Score (NPS) categories

Net Promoter Score is a widely used metric that measures the extent to which a person or group would recommend a product or service. In our case, learners are asked the extent to which they would recommend the course to a colleague, on a scale from 0-10. 

Responses by each learner are broken into the three categories listed below. To obtain the overall NPS for the course, you subtract the percentage of detractors from the percentage of promoters. NPS scores range from -100 to +100.

• Promoter: The learner gave a rating of 9-10.
• Neutral: The learner gave a rating of 7-8.
• Detractor: The learner gave a rating of 0-6.

Here is an example to illustrate how the calculation is made:

• Cohort(s) size: 12 learners
• Promoters: 6 learners (i.e. 50%)
• Neutral: 2 learners (i.e. 17%)
• Detractors: 4 learners (i.e. 33%)

The NPS score for this group would be +17, stemming from 50-33 (% Promoters - % Detractors).

This article provides the data and associated ranges for the Engagement metric in the HILA report.

• Learn how to generate a report using the High Impact Learning Dashboard.
• Learn about the data category ranges within the Advocacy metric.
• Learn about the data category ranges within the Understanding metric.
• Learn about the data category ranges within the Commitment metric.

Our Engagement measure looks at both the participation of learners going through the course - whether they proceeded from activity to activity - but also whether they actively used the discussion forums to exchange ideas with others in their cohort, both in the quantity of discussion, as well as the quality. In this way, the Engagement metric is much richer than most, as it combines both the learners' completion as well as their exchange of ideas.

Engagement category ranges

• Very High: 61-100
• High: 40-60
• Moderate: 26-40
• Low: 11-25
• Very Low: 1-10
• No Show: 0

Participation

• On Track: The learner has completed greater than 70% of required course activities.
• Behind: The learner has completed 70% or less of required activities but above 0%.
• No Show: The learner has not completed any required activities, i.e. 0%.

Discussion categories

• Silent: A learner in this category did not post a comment or reply to other learners’ posts in the course’s discussion activities.
• Poster: A learner in this category posted in one or more discussion activities but did not respond to other learners’ posts.
• Responder: A learner in this category responded to one or or more posts from other learners but did not post a comment in the course’s discussion activities.
• Collaborator: A learner in this category both posted a comment and responded to other learners’ posts in the course’s discussion activities.