Insights

This article outlines the features and functionality of the Team Plan User activity dashboard, which provides admins with a macro view of how active learners are in their Udemy Business accounts. 

Please note: an expanded version of this feature with data export reports is available with the Enterprise Plan.

• Enterprise Plan account: learn how to access your user activity.

What information and metrics are on the Team Plan User activity dashboard?

To view the User Activity dashboard, please navigate to your Admin Overview page. From there, select User Activity from the left-hand side panel.

Note: To navigate back to the learner homepage with courses and content, click on the logo on the top left hand corner of your logged in account. 

Some of the user activity metrics that are featured on this dashboard include total activity, average activity per user, average activity per active user, and active user base. In addition, the activity trend chart and the most active users are listed here. 

• Review what defines an active user so their learning is captured in Insights and reporting.
• Learn more about the difference between activated and deactivated users and how it impacts admin reporting.

How are the minutes spent calculated compared to the Activity trend?

The Activity trend line is an approximation of the Minutes spent line where the spikes are smoothed out so that the general trend can be more easily visualized. This is done using a regression algorithm which calculates a moving average of the minutes spent to plot the Activity trend line.

When is data in the User activity dashboard updated?

Data in the User activity dashboard is updated regularly. 

• Learn more about when data is updated.

Udemy Business includes our Badge Insights dashboard, which provides admins or group admins with an aggregated view of what badges your learners are working towards, or have imported into your Udemy Business account. 

This enables you to easily identify, at a glance, the number of users within your organization who have validated their skills by earning a third party certification, as well as track how many learners are progressing towards their certifications within Udemy Business.

How to access the Badge Insights dashboard

Admins or group admins can access the Badge Insights dashboard by moving their cursor to Manage at the top of the page, and then by selecting Insights and reporting and then Badge Insights.

The data obtained from the imported badges are also available via APIs. For more information on integrating with the Badging APIs, input https://<company name>.udemy.com/dev-portal/api-reference/, with your organization’s domain reference.

Reviewing a specific learners’ certification badge achievements

Learn how admins and group admins can view their learners’ certification badge achievements.

This article answers frequently asked questions about the Skills Benchmarks feature, available in the Skill Insights dashboard. 

• Learn about the Skill Insights dashboard

Frequently asked questions

• What are Skills Benchmarks?
• Where can I access the feature?
• What are the most common use cases for the data?
• Why does this data matter?
• Who is this report for?
• Why do benchmarks not appear in my dashboard? 

What are Skills Benchmarks?

Skills Benchmarks is a Skill Insights dashboard feature that allows admins and group admins to view popular skills within their organization and compare those skills to industry benchmarks.

By surfacing benchmarks on industry trends, customers can see how they’re tracking against other companies across a range of skills. 

Where can I access the feature?

Skills benchmarks can be accessed alongside other insights in the Skill Insights dashboard (Learn more).

What are the most common use cases for the data?

Some of the primary uses for this data include: 

• Redirect your employees toward more relevant learning
• Understand the skill levels within your organization and how that compares to your industry
• See what skills gaps you should address in your learning strategy

Why does this data matter?

With Skills Benchmarks, you can compare your  Udemy Business learning trends, not only within your own organization, but also across other companies in your industry.

Who is this report for?

The dashboard is for all admins and group admins. 

Why do benchmarks not appear in my  dashboard? 

We aim to provide the most accurate and meaningful comparisons when it comes to providing benchmarks. To make sure we are delivering the most valuable insights, and to maintain data privacy and integrity, Udemy provides anonymized, amalgamated benchmarks if more than 5 companies fulfill the comparison group criteria of :

• Industry
• Company size
• Region/country

This article outlines important information regarding our Skill Insights dashboard, which visualizes the skills of your company based on the content that you are consuming/ learning. 

Please note:

• This feature is only available with the Enterprise Plan.
• Custom courses created in the course creation tools will not show up in the Skill Insights Dashboard. Learner activity and progress for custom courses can be viewed in admin data reports.

Overview of the Skill Insights dashboard

With the Skill Insights dashboard, you are able to understand your employees’ learning priorities, by drilling into categories and topic data to see where exactly their time is being spent. This knowledge can help you drive learning in a direction that helps you reach your organization’s goals and to see if your employees' learning aligns with your business goals.

The dashboard’s features include:

• A breakdown of the percentage of time spent learning business vs. tech vs. personal development skills.
• Total count of topics in business, technology and personal development that the organization's learning spans.
• An interactive chart that provides insights into the percentage of time spent learning and the total time spent learning by category, subcategory and topic. The side panel also shows the top courses for each category, subcategory and topic so that you can easily assign or recommend these to other learners in your organization.
• A chart with the most popular business, technology and personal development topics in your organization ranked by time spent watching. The chart also details what percentage of total learning these top topics account for.
• The dashboard allows you to view data by all users or to dive into specific groups or sections. Additionally, you can look at learning data for specific time frames. 
• Priority Skills - Each admin can set their Priority Skills which will facilitate tracking and reporting on the sub-set of skills that are most important to you. You will have the opportunity to understand skill gaps and to take targeted actions when needed.
• Skills Benchmarks - Admins and group admins can view popular skills within their organization and compare those skills to industry benchmarks. By surfacing benchmarks on industry trends our customers can see how they’re tracking against other customers across a range of skills.
  • View our frequently asked questions about Skills Benchmarks.
• The ability to export a User skills activity report, which provides detailed skills activity per user in a CSV format. 
  

Who is the Skill Insights dashboard for?

The dashboard is for all admins and group admins. Group admins will only be able to see skill insights for their groups.

How is time spent learning attributed to topic/category/sub-category?

The pie chart has four levels:

• Business vs Technology vs Personal Development: Business includes all categories containing courses that are primarily focused on business; technology includes all categories containing courses primarily focused on technology topics, and personal development includes all categories containing courses that are primarily focused on personal development.
• Categories: These are the categories shown in the Explore/Browse experience. 
• Subcategories: These are the subcategories shown in the Explore/Browse experience. 
• Topics: Courses are tagged with topics. Some courses have one primary topic and all learning in the course is attributed to this primary topic. Other courses have more than one primary topic. In these cases, we attribute all learning in the course to each of the topics.

A course can exist within more than one subcategory and courses with the same primary topic can also exist in different subcategories. As a result, this means that learning attributed to a single topic can be split across multiple subcategories. 

The top topics list includes all learning attributed to the topic regardless of which subcategory it belongs to.  

Which learning categories make up each classification?

The categories include multiple topics, some of which are:

Tech categories:

Cloud Computing

Cybersecurity 

Design

Development

IT Operations

Data Science

Business categories:

Finance & Accounting

Workplace & Human Resources

Leadership & Management

Marketing

Project & Product Management

Sales

Personal Development categories:

Personal Development

The Enterprise Plan includes our Course Insights Dashboard, which provides you with a comprehensive view of the courses your users are learning. It also enables you to discover the most popular courses in your organisation, based on Active Users, Average Minutes Watched (per active user), enrolled users, assigned users and ratings provided by learners within your organization. 

Please note: this feature is only available with the Enterprise Plan.

Sorting and filtering Course Insights

You can sort by the number of Active Users, the Average Minutes Watched (per active user), the number of enrolled users or the number of assigned users, depending on which metric is most important to you.

You can use the filters to discover popular courses within the following: 

• groups 
• sections
• time ranges
• categories

For example, you can use the time range filter to see how many courses were active and how many users were active in each course within the Data Science category during the last 30 days. You can choose time ranges with day granularity within the last 120 days. Prior to this, monthly time ranges can be selected. 

Definition of active 

A course is active if there is at least one active user in it. Learn more about what actions are considered activity for users in Udemy Business.

While the time-range filter on this dashboard enables you to see course activity within the selected time range, the average ratings shown on it are the ratings given by your users over all-time (since June 30th 2020), regardless of what time range is selected. This enables you to see the overall rating for courses that are currently trending at your organization.

Viewing learner ratings and reviews

From the Course Insights dashboard you can navigate into each course to discover which users are enrolled and to see any ratings and reviews that learners at your organization have left on the course.

Viewing Learner Activity

From the Course Insights dashboard, you can click into any course to view learner activity. Important: When a learner enrolls in a course or is assigned a course, they will appear on this page in realtime. Their activity data, however, won’t be shown until the next data refresh. To learn about how frequently data is refreshed, click here.

We want to empower our Udemy Business admins with actionable insights to help guide and optimize their learning programs. As an admin, through our ratings and review dashboard, you will not only be able to see how your employees have rated courses they’ve taken in Udemy Business, but also any reviews employees left on the course as well. We encourage you to utilize learner sentiment about courses to continuously make improvements to your course recommendations, the content within learning paths, and course selections for specific programs, etc.

How to access the Ratings and Reviews dashboard

1. Click on the Manage menu in the top right of the screen.

2. On the right side, go to Insights and Reporting. Then, select Ratings and reviews.

Top 5 rated courses

This section will show the top 5 rated courses at your organization as well as the average internal rating for those courses. This will help you to understand well-received courses at a glance and the general sentiment about the courses  from your learners. 

Courses must have at least 3 ratings and have been active within the last 30 days to be included. (If there are none eligible in the last 30 days then the top 5 over the last 90 days is shown.)  

Recent ratings and reviews

This section contains all the ratings and reviews from learners at your organization (from June 30, 2020). You can filter them by:

• Date range
• User name
• Groups
• Sections
• Rating. 

Ratings are sorted by most recent first. This will help you to understand the overall sentiment learners have about courses. You can also export a ratings report by using the Export tool.

Course Insights dashboard

Rating information is also included on the Course Insights dashboard where the average internal rating for each course is shown (when there are at least 3 ratings for the course).

You can filter by the category enabling you to discover the highest-rated courses for a specific subject.  

More information is available in the Course Insights article.

Group admin access

Group admins currently do not have access to ratings and review dashboards.

How are internal course ratings calculated?

The average internal rating is calculated by the sum of all ratings from employees within the organization, divided by the total number of employees that rated the course.

We calculate and show the average internal rating for a course only when at least 3 ratings have been provided by learners at your organization. 

Ratings and review data is only included from June 30th 2020. 

NOTE* The internal rating is representative of only your organization and is calculated differently from the course rating that learners see, which includes ratings from all learners on Udemy. 

Data freshness

The data freshness notice shows the latest time when all data has been last updated. The Top 5 rated courses section is up to date at this time, however, the Recent Ratings and Reviews section is updated in real-time. 

Our User Adoption Funnel not only provides you with a view into how people are adopting learning, but also gives you the power to take direct action to get people to sign up, enroll in a course and start watching their first lecture.  

How is adoption defined?

While adoption can mean different things for different organizations, at Udemy Business we define ‘adoption’ as the point at which users have watched their first lecture, after having joined and enrolled in a course.

The Organization Adoption Rate is the number of users that have watched their first lecture as a percentage of the total number of licenses you have purchased.

We present your user adoption in a funnel because it best describes the different steps users must take from when they first get access to Udemy Business until they reach the point where they’re learning, ie. watching their first lecture.

We highlight the steps of the funnel where there is a drop off in users joining, enrolling or watching a lecture to help you understand which users are not yet adopting Udemy Business and help you take action so you can encourage them to get started.

This report is in real-time and shows data since the creation of your organization's account.

Licenses

This is the total number of licenses you have purchased for your employees on Udemy Business.

Invitations Sent

This column shows you the number of invitations you have sent to your team. It includes users who have either accepted an invitation and joined your Udemy Business account or still have a pending invitation. Cancelled invitations are not included.

Details of invitations sent, and users who have not accepted their invitations is not shown for accounts that have SSO login enabled.

Joined

These users have accepted their invitation to join your Udemy Business account and have already logged in at least once.

Users that drop out of the adoption funnel at this step are the users that have not accepted their invitations yet. You can use the Email These Users link to remind them to claim their account, or  the Manage Invitations link in the report for quick access to your pending invitations list.

Enrolled

These users have enrolled in at least one course. The course may have been assigned to them or it may have been something they found and selected themselves.

Users that drop out of the adoption funnel at this step are the users that have logged in at least once but have not yet enrolled in a course. You may wish to assign or recommend a course to help them get started. Or you can use the Email These Users link to remind them to start learning or export a list of these users and send it to the appropriate manager to follow up.

Watched

These users have watched their first lecture of one of the courses they have enrolled in.

Note: watched activity will be only reported when a user views and completes a video lecture. Completing other types of lectures, like quizzes, practice tests, coding exercises etc., will not prompt any activity.

If a user unenrolled from a course, then any watched activity that was recorded, will be removed from reporting.

Users that drop out of the adoption funnel at this step have enrolled in at least one course, but have not yet completed their first lecture. It might be worth sending them some encouragement. You can use the Email These Users link to remind them to start learning or Export a List of these users and send it to the appropriate manager to follow up.

Adoption by Groups or Sections

If you have groups or sections set up you can see how adoption is going for a specific group, set of multiple groups, or sections, by selecting it from the filter at the top of the page.

The Group Adoption Rate is the number of users (in a group) that have watched their first lecture as a percentage of the users who have joined your account.

SSO-enabled organizations

If SSO is enabled in your organization, the adoption funnel will contain a scaled down version of the metrics provided in the Adoption Funnel.

Data on numbers of users joined, enrolled and watched will be displayed. Data on licenses and invitations is currently not displayed, as users are provisioned via SSO.

We encourage you to also view the User Activity and Course Insights Dashboards (not available for Team Plan customers) to get additional insight on your users’ learning and engagement activity. 

This article outlines the features and functionality of the User activity dashboard, which provides admins a macro view of how active learners are in their Udemy Business accounts. 

• Team Plan account: learn how to access your User activity dashboard

What information and metrics are on the User activity dashboard?

To view the User Activity dashboard, please navigate to your Admin Overview page. From there, select User Activity from the left-hand side panel.

Some of the user activity metrics that are featured on this dashboard include total activity, average activity per user, average activity per active user, and active user base. In addition, the activity trend chart and the most active users are listed here. Admins can review activity metrics in visual format as well as export user activity and trends via reports.

• Review what defines an active user so their learning is captured in Insights and reporting.
• Learn more about the difference between activated and deactivated users and how it impacts admin reporting.

If you have groups or sections set up you can view user activity for a specific group or section by selecting the group from the filter at the top of the page.

What exports are available on the User activity dashboard?

There are three options for exports on the User activity dashboard:

1. User activity report (active users) - This report includes activity data for all activated users who have been active in their account in the selected time frame. In order to be considered active, a user must enroll in a course, watch lectures, or complete an assignment or quiz. If a learner was active during the selected time frame and later deactivated, they will be included on the report.
2. User activity report (all users) - This report includes activity data on all activated users in an account, including deactivated users and users who weren’t active in the selected time frame.
3. User activity trend report - This report includes data on minutes watched by each user for each day in the selected time frame.

Learn more about the difference between activated and deactivated users in admin reports.

How are the minutes spent calculated compared to the Activity trend?

The Activity trend line is an approximation of the Minutes spent line where the spikes are smoothed out so that the general trend can be more easily visualized. This is done using a regression algorithm which calculates a moving average of the minutes spent to plot the Activity trend line.

When is data in the User activity dashboard updated?

Data in the User activity dashboard is updated regularly. 

• Learn more about when data is updated.