This article discusses the difference between activated and deactivated users in Udemy Business accounts, and how this impacts admin reporting.
What is the difference between an activated user and a deactivated user?
Activated users are users that currently hold a license and have joined their Udemy Business account. This means they have logged in or authenticated via Single Sign On (SSO) at least once to activate that license. Users who have not activated their license are not included in any insights and reporting.
- Learn more about what defines an active user so their learning is captured in Insights and reporting.
Deactivated users are users who have had their licenses removed and can no longer access the Udemy Business portal. Deactivated users retain all data relating to their learning, however, and can be reactivated at any time by the admin.
Deactivated users who have their Personally Identifiable Information (PII) removed
Deactivated users can have their Personally Identifiable Information (PII) permanently removed for General Data Protection Regulation (GDPR) compliance, or a similar purpose.
Once a user’s PII data has been deleted they become anonymized users and all references to them in Udemy Business are labeled with “Anonymized User”. Please note that deleting a user’s PII is irreversible.
This article includes answers to some of the frequently asked questions our team receives regarding data export reports, and what information is featured in certain columns.
For steps on how to generate and download a report, please click here.
- What information is included in the Minutes Video Consumed Column of User Activity and User Course Activity Reports?
- Which specific result is posted in the Final Results for Quizzes column of User Progress Reports?
- What information is provided in the User Groups Column?
- What does the First Date Complete column of the User Course Activity report indicate?
- How is the Course Duration column calculated in the User Course Activity Report?
- What does the Practice Activities Completed Successfully column indicate in the User Course Activity Report?
- Do reports include the course progress of unenrolled users?
- When is data updated? And what timezone are the data exports in?
- How can a learner’s certificate of completion have a date/timestamp that is different from the completion date/timestamp entry for that course in admin reports?
What information is included in the Minutes Video Consumed Column of User Activity and User Course Activity Reports?
The Minutes Video Consumed column in the User Course Activity and User Activity reports
- Include rewatches
- Do not include any parts of a lecture that are skipped
- Do not take into account video playback speed.
Rewatches: If a user watched 1 hour of the course on Monday, and then on Tuesday, they watched the same 1 hour once again. In this case, the user spent two hours learning on Udemy Business that week, and this would be reflected in both reports.
Playback Speed: If the user watched 2 hours of content in 1 hour, by viewing the content at 2x the speed, the report would reflect they spent 2 hours viewing the course.
Skipped Parts of a Lecture: if a user starts watching a 10 minute lecture and after 1 minute, they skip to the end of the lecture, we will only report the actual time they spent on the lecture, in this case about 1 minute. The lecture will however be marked as complete.
The Minutes Video Consumed column only includes time that users spent watching video lectures, and not time they spent reviewing lectures that do not include videos. Lectures that do not include videos, however, still count towards course completion. For example, a User Activity Report could indicate that a user has completed four lectures, but there is no time recorded in the Minutes Video Consumed column, if those four lectures were not video lectures.
Which specific result is posted in the Final Results (Column N) for the Quizzes column of User Progress Reports?
The Final Result (if Quiz) column (column N) of the User Progress Reports includes the score(s) a user recorded completing a quiz. For example, if a user takes a quiz twice and receives 80% on their first try, and then receives 90% on their second and final attempt, then the User Progress Report will contain two rows. Each row will include the score the user recorded for that respective attempt. Please note that there is no limit to how many times a user can retake a quiz.
What information is provided in the User Groups Column?
Data Export Reports feature a User Groups column which display all groups that a user is part of. Admins and group admins are able to easily see group membership details when exporting user reports, including 'all users' within a group. Groups are not present in the Reporting API, but customers can use our User API to get group data. In addition, when exporting reports, the list of groups is scrollable for easier selection.
What does the First Date Complete column of the User Course Activity report indicate?
User Course Activity Reports feature a First Date Completed column, which records the date the course was first completed by the learner. As a result, if the instructor adds new lectures or content, the date and time stamp won't be lost. If the learner completes the extra content it will be recorded as "Date Completed". The column is present in both the CSV and the Reporting API.
How is the Course Duration column calculated in the User Course Activity Report?
The course duration column in the user course activity report indicates how many minutes of video content the course includes. As a result, practice test courses may have a 0 or a small number listed in the course duration column, since practice tests typically feature little to no video content.
What does the Practice Activities Completed Successfully column indicate in the User Course Activity Report?
The No. of Practice Activities Completed Successfully column in the user activity report indicates how many practice activities the user has completed successfully. Practice activities include quizzes, coding exercises, and practice tests. (If the practice activity was started but not completed or was completed unsuccessfully it will not be included in this count.)
If there is a row in the User Activity Report with 0 minutes consumed and 0 practice activities complete, this indicates that the user started but did not complete one or more practice activities. Or that they started and completed one or more practice activities unsuccessfully.
Do reports include the course progress of unenrolled users?
If a user unenrolls from a course, or their enrollment is transferred from another account, including an account at the Udemy marketplace, some activity data will be reflected in the User Course Activity report.
Many of the report fields, however, are dependent on the original enrollment record and will therefore have no data if a learner unenrolls. Specifically, unenrolling can affect data such as “% Marked Completed”, “Date Enrolled” and “Date Completed” in the User Course Activity report.
If you have any questions regarding this please contact us.
When is data updated? And what timezone are the data exports in?
How can a learner’s certificate of completion have a date/timestamp that is different from the completion date/timestamp entry for that course in admin reports?
The certificates of completion that learners receive are localized to the timezone of the IP address of the learner's device at the time the course is completed. The admin reports, however, are in standard UTC format. Therefore the date/timestamps may be different, but they will always equate to the same date and time.
For example, a learner located in San Francisco, California, U.S.A. completes a course at 9:00pm / 21:00 Pacific Standard time on the 25th of February. When we convert to UTC standard format, it would equate to 5:00am / 05:00 on the 26th of February. So, that is the date/timestamp that would be listed in the data reports.
The Group Membership Log includes extensive information about what actions a group has completed and what employees have been added or removed.
Admins can access the Group Membership log of every group. Group admins, however, can only access the Group Membership Log for the groups that they are group admins for.
For more information on how to create user groups, please click here.
How to access the group membership log
The Group Membership Log can be accessed by doing the following:
- From the Manage dropdown menu at the top, select Manage Users
- On the left hand side of the page, click on Manage Groups
- Click the ellipses to the right of the group name. Or, after selecting a specific group, click on the ellipses beside the Invite new users button.
- Select Group Membership Log
What information does the Group Membership Log include?
The log reports on:
- Date: Date action was completed
- Name: Name of employee that was added or removed
- Email: Email of employee that was added or removed
- Action: Action performed
- Group: Group that log is reporting on
- By: The method in which the user was added/removed from the group - i.e. SSO data, CSV upload, name of admin/group admin, etc.
Note: The Group Membership log includes data from 23 March 2018 onwards.
This article explains the reports admins can generate and download to review learner activity and progress.
Data export reports that are available for download
Under Manage --> Insights and Reporting --> Data Export Reports, there are three reports available for admins to download.
- User Activity Report: Get information at the user level (e.g. how many courses has each user enrolled in, learning time per user, etc). View your top learners across your company or by specific group.
- User Course Activity Report: Get information at the enrollment level. Understand the courses individuals and groups are learning and sort by active users, minute per active user, and enrollments.
- User Progress Report: Get information at the lecture level. This is our most granular report and you'll see quiz results for example. This report will include any lectures or quizzes that were either started or completed within the time range selected (or across all time if none is selected).
These reports and others are available for download via other Insights and Reporting dashboards.
How to generate and download data export reports
To download your data export report choose the users, groups or sections you wish to include and the timeframe and click Export. The data will be up to date as per the data freshness notice at the top of the page.
Note: To ensure that downloads are successful, we cap the number of records exported at 10 million records. Please choose a suitable filter to avoid reports getting capped. You will be presented with a notification if there is a likelihood of your report getting capped.
Once your report is ready, you will receive an email notifying you of this. The report will also be listed under the Recent Reports section of the page.
Deleting a Report
If you wish to delete a report you have previously exported, you can do so by following these steps:
- Scroll down the Data Export Reports page to Recent Reports
- Click on the trashcan icon beside the report you wish to delete
- Confirm you wish to delete the report
Admins and group admins can download three data export reports to view deeper insights into learning paths activity.
Path activity report: provides detailed path activity for all public paths. Some of the fields in the report include Path ID, Path Title, Path total steps, No. of Users Enrolled and Total Consumption in Minutes among others. Available on the Path Insights overview page.
Path user activity report: provides details on enrollments and activity per user for the path listed. There will be one entry per user enrolled in the path, tracking their overall path consumption.
Some of the fields in the report include User First Name, User ID, User Groups, Path Assigned Due Date and Path Date Completed among others. Available on the Path Insights page for a specific path.
Path item activity report: provides deeper insights into item activity per user for the path listed. There will be one entry per completed item per user, detailing the individual item consumption.
Some of the fields in the report include Item section title, Path item ID, Path item object type, Path item object ID and Path item duration among others. Available on the Path Insights page for a specific path.
Path insights reports can be filtered by users, groups, or sections.
To download the path activity report, navigate to Insights and reporting > Path Insights > Export.
To download the path user activity and path item activity reports, click on a specific path > Export.
Who can access and download the reports?
Only admins and group admins have access to the Path Insights page and the ability to download these reports. If a path is private, admins and group admins must be the editor or enrolled in the path to be able to view Path Insights for the path and download these reports.
Additionally, the Path User Activity Report and Path Item Activity Report are available on the Path Insights page for a specific path. For private paths, as long as the admin or group admin is enrolled in, or is the editor of that particular path, they will be able to download these reports.
Which learners’ data is included?
Important: path reports only contain data for learners that are currently enrolled in respective learning paths. If a learner unenrolls from a learning path, any previous data from their progress will no longer be included in the reports. Once a learner re-enrolls in the path, their data will be restored to the reports after the next data refresh.
Microsoft Excel has local settings that can cause formatting errors when viewing Data Export Reports from your Udemy Business account in the program. This article includes troubleshooting resources you can use to resolve some of these formatting issues.
Import data through Excel
The Udemy Business data exports support multiple delimiters (comma, tab, semi-colon etc.), in an attempt to resolve any issues you may encounter when viewing data exports in Microsoft Excel.
If you still find that your Data Export Reports are misformatted when viewed in Excel (i.e. all entries are in one row with no columns, separated by commas), please follow the steps outlined in this Microsoft article. It includes instructions on how you can import data through Excel, so that the report appears correctly.
Additionally, Excel may have autofit settings enabled that can cause data cells to appear ‘squished’. If you need to adjust the column width or row height of the Excel file, please review this Microsoft article.
If you have questions regarding the steps outlined in the articles above, or continue to have formatting issues after attempting the steps, we recommend contacting Microsoft Support.
Once you've followed these steps, please try to download and open the reports again. You should be able to see them as expected.
If you have any additional questions or continue to encounter issues, you can also contact us here.
This article discusses how updates to the Udemy Business course collection may impact Insights and Reporting for admins and group admins.
If a course is no longer available in the Udemy Business collection, is prior learning data deleted from admin dashboards and reports?
All learning data is retained, even if a course is no longer available in the Udemy Business collection. Admins are able to view more detailed information from the Course Insights dashboard, or on individual learner’s profile pages.
The minutes consumed and other activity on these courses will also still be recorded via admin Insights and Reporting.
We want to empower our Udemy Business admins with actionable insights to help guide and optimize their learning programs. As an admin, through our ratings and review dashboard, you will not only be able to see how your employees have rated courses they’ve taken in Udemy Business, but also any reviews employees left on the course as well. We encourage you to utilize learner sentiment about courses to continuously make improvements to your course recommendations, the content within learning paths, and course selections for specific programs, etc.
How to access the Ratings and Reviews dashboard
To access the Ratings & Reviews dashboard hover over the Manage menu, select Insights and Reporting and then Ratings & Reviews on the left-hand side of the page.
Top 5 rated courses
This section will show the top 5 rated courses at your organization as well as the average internal rating for those courses. This will help you to understand well-received courses at a glance and the general sentiment about the courses from your learners.
Courses must have at least 3 ratings and have been active within the last 30 days to be included. (If there are none eligible in the last 30 days then the top 5 over the last 90 days is shown.)
Recent ratings and reviews
This section contains all the ratings and reviews from learners at your organization (from June 30, 2020). You can filter them by:
- Date range
- User name
Ratings are sorted by most recent first. This will help you to understand the overall sentiment learners have about courses. You can also export a ratings report by using the Export tool.
Course Insights dashboard
Rating information is also included on the Course Insights dashboard where the average internal rating for each course is shown (when there are at least 3 ratings for the course).
You can filter by the category enabling you to discover the highest-rated courses for a specific subject.
More information is available in the Course Insights article.
Group admin access
Group admins currently do not have access to ratings and review dashboards.
How are internal course ratings calculated?
The average internal rating is calculated by the sum of all ratings from employees within the organization, divided by the total number of employees that rated the course.
We calculate and show the average internal rating for a course only when at least 3 ratings have been provided by learners at your organization.
Ratings and review data is only included from June 30th 2020.
NOTE* The internal rating is representative of only your organization and is calculated differently from the course rating that learners see, which includes ratings from all learners on Udemy.
The data freshness notice shows the latest time when all data has been last updated. The Top 5 rated courses section is up to date at this time, however, the Recent Ratings and Reviews section is updated in real-time.
Our User Adoption Funnel not only provides you with a view into how people are adopting learning, but also gives you the power to take direct action to get people to sign up, enroll in a course and start watching their first lecture.
How is adoption defined?
While adoption can mean different things for different organizations, at Udemy Business we define ‘adoption’ as the point at which users have watched their first lecture, after having joined and enrolled in a course.
The Organization Adoption Rate is the number of users that have watched their first lecture as a percentage of the total number of licenses you have purchased.
We present your user adoption in a funnel because it best describes the different steps users must take from when they first get access to Udemy Business until they reach the point where they’re learning, ie. watching their first lecture.
We highlight the steps of the funnel where there is a drop off in users joining, enrolling or watching a lecture to help you understand which users are not yet adopting Udemy Business and help you take action so you can encourage them to get started.
This report is in real-time and shows data since the creation of your organization's account.
This is the total number of licenses you have purchased for your employees on Udemy Business.
This column shows you the number of invitations you have sent to your team. It includes users who have either accepted an invitation and joined your Udemy Business account or still have a pending invitation. Cancelled invitations are not included.
Details of invitations sent, and users who have not accepted their invitations is not shown for accounts that have SSO login enabled.
These users have accepted their invitation to join your Udemy Business account and have already logged in at least once.
Users that drop out of the adoption funnel at this step are the users that have not accepted their invitations yet. You can use the Email These Users link to remind them to claim their account, or the Manage Invitations link in the report for quick access to your pending invitations list.
These users have enrolled in at least one course. The course may have been assigned to them or it may have been something they found and selected themselves.
Users that drop out of the adoption funnel at this step are the users that have logged in at least once but have not yet enrolled in a course. You may wish to assign or recommend a course to help them get started. Or you can use the Email These Users link to remind them to start learning or export a list of these users and send it to the appropriate manager to follow up.
These users have watched their first lecture of one of the courses they have enrolled in.
Note: watched activity will be only reported when a user views and completes a video lecture. Completing other types of lectures, like quizzes, practice tests, coding exercises etc., will not prompt any activity.
If a user unenrolled from a course, then any watched activity that was recorded, will be removed from reporting.
Users that drop out of the adoption funnel at this step have enrolled in at least one course, but have not yet completed their first lecture. It might be worth sending them some encouragement. You can use the Email These Users link to remind them to start learning or Export a List of these users and send it to the appropriate manager to follow up.
Adoption by Groups or Sections
If you have groups or sections set up you can see how adoption is going for a specific group, set of multiple groups, or sections, by selecting it from the filter at the top of the page.
The Group Adoption Rate is the number of users (in a group) that have watched their first lecture as a percentage of the users who have joined your account.
If SSO is enabled in your organization, the adoption funnel will contain a scaled down version of the metrics provided in the Adoption Funnel.
Data on numbers of users joined, enrolled and watched will be displayed. Data on licenses and invitations is currently not displayed, as users are provisioned via SSO.
This article outlines the features and functionality of the User activity dashboard, which provides admins a macro view of how active learners are in their Udemy Business accounts.
Please note: this feature is only available with the Enterprise Plan.
What information and metrics are on the User activity dashboard?
To view the User activity dashboard, select Insights and reporting on the Manage dropdown menu at the top, and then User activity on the left-hand side.
Some of the user activity metrics that are featured on this dashboard include total activity, average activity per user, average activity per active user, and active user base. In addition, the activity trend chart and the most active users are listed here. Admins can review activity metrics in visual format as well as export user activity and trends via reports.
- Review what defines an active user so their learning is captured in Insights and reporting.
- Learn more about the difference between activated and deactivated users and how it impacts admin reporting.
If you have groups or sections set up you can view user activity for a specific group or section by selecting the group from the filter at the top of the page.
What exports are available on the User activity dashboard?
There are three options for exports on the User activity dashboard:
- User activity report (active users) - This report includes activity data for all activated users who have been active in their account in the selected time frame. In order to be considered active, a user must enroll in a course, watch lectures, or complete an assignment or quiz. If a learner was active during the selected time frame and later deactivated, they will be included on the report.
- User activity report (all users) - This report includes activity data on all activated users in an account, including deactivated users and users who weren’t active in the selected time frame.
- User activity trend report - This report includes data on minutes watched by each user for each day in the selected time frame.
Learn more about the difference between activated and deactivated users in admin reports.
How are the minutes spent calculated compared to the Activity trend?
The Activity trend line is an approximation of the Minutes spent line where the spikes are smoothed out so that the general trend can be more easily visualized. This is done using a regression algorithm which calculates a moving average of the minutes spent to plot the Activity trend line.
When is data in the User activity dashboard updated?
Data in the User activity dashboard is updated regularly.
This article outlines how organizations can ensure offline course viewing is reflected in their reporting.
How to verify offline course viewing
In order for any offline consumption to be reflected in Admin Insights and Reporting, learners must take the following actions:
- Bring their mobile device back online by connecting to wifi or cellular data.
- Open up the Udemy Business app again once the device is online to allow their progress to sync.
This article discusses how frequently Udemy Business insights and reporting data is updated.
How frequently are dashboards and data exports updated?
The User Adoption funnel is updated in real time. The Path Insights dashboard is updated once per day. The User Activity, Ratings & Reviews, and Course Insights dashboard, as well as all Data Export Reports, are updated several times per day.
How can I tell how fresh the data is?
The date and time up to which data is included is indicated by a data freshness notice in the top right of each dashboard. The timestamp is localized based on your computer’s local settings.
What timezone are the data exports in?
All datetime stamps within Udemy Business data export reports are localized into a Coordinated Universal Time (UTC) format. The localized time is based on the Admin’s local computer timezone setting that exported the report.
Please note: the time frame filters on the data exports reports page are based on the UTC -8 time zone (Pacific Standard Time). Depending on your location, and what dates you are selecting, you may need to update the filter accordingly to source the required data.
For example, admins in the UTC+9 time zone (i.e., Japanese Standard Time) wishing to retrieve data for their organization’s learners that occurred on 08/01, would need to select 07/31.
Review what reports and dashboards are available to help admins understand how your company’s people are learning.
This article explains what actions learners must take in their Udemy Business account in order to be considered ‘active’, so that their learning is captured in Insights and Reporting for admins and groups admins to review.
Learner engagement in Udemy Business: What activity will be recorded
The following actions are considered activity in Udemy Business and will be recorded within Insights and reporting for admins and group admins:
- Enrollment of a course (either self-enrolled or assigned)
- Progress through a video lecture (in chunks of 15 seconds)
- Completion of a lecture
- Attempt of a practice activity
- Completion of a practice activity
- Completion of a course
- Reviewing/rating a course
So long as a user has taken one of the above actions during the selected time frame, they are considered active and will be included in data export reports.
What is recorded in User activity dashboard and exports
When viewing the User activity dashboard the number of active users is counted as a percentage of the total number of users who were licensed in the account during the time range selected. This includes any users that joined (visited for the first time) before the end of the time range. It may also include deactivated users, if they were deactivated after the start of the time range. By calculating it this way, we ensure that all activity that occurred during the time range is included in the calculations.
- The Active User Base pie chart that is located under the User Activity dashboard, includes users who have been active in their Udemy Business account by taking at least one of the actions listed above in the selected timeframe. The number of active users shown under the Active User Base pie chart should match the count of users shown in the User Activity report.
- The User Activity Report (active users), which can be downloaded from the User Activity dashboard, includes any user who was active in their account during the selected time frame. Any users that are currently deactivated at the time of export will be marked deactivated in the report.
- The User Activity Report (All Users), which can also be downloaded from the User Activity dashboard, includes all users in the account, regardless of if they were active during the selected time range. This enables you to identify users that have not been active so that you can take steps to re-engage them in learning.
Activated versus deactivated users
Learn more about the difference between activated and deactivated users and how it impacts admin reporting.
This article outlines what dashboards and reporting features are available in your account’s Insights and reporting, which you can use to understand how people in your company are learning. You’re able to view information for your entire organization or select specific groups to understand learning trends.
How to access Insights and reporting
To access the reporting features in your Udemy Business account, hover your cursor over Manage and select Insights and Reporting from the dropdown menu.
The available Insights and reporting options
From the menu on the left-hand side of the Insights and reporting page, you’ll find the following options:
Skill Insights: This dashboard will show you the skills your employees are investing time in learning, and provides a clear picture of time spent learning per category, subcategory and topic. As a leader, you will be able to better define learning paths for different teams and understand areas of opportunity for developing skills within your teams. Learn more about the Skill Insights dashboard.
Group insights (Beta): The Group insights dashboard provides insights into which groups in your organization are the most engaged in learning, and which may need more support and guidance. Learn more about Group insights.
Please note: This dashboard is currently a Beta feature and so may not yet be available in your Udemy Business account.
User Adoption Funnel: This dashboard will allow you to see how many and which users in your organization have adopted the Udemy Learning account and have started using it. The dashboard is also available to Team plan admins. Learn more about the User Adoption Funnel.
User Activity: This dashboard will help you understand more about user activity within your Udemy Learning account. Learn more about the User Activity dashboard.
Course Insights: This dashboard will help you to understand more about what users are learning within the Udemy Learning account. Learn more about the Course Insights dashboard.
User Progress: This report is only available as a download from the Data Export Reports page and the individual Course details page (when you click into a course on the Course Insights dashboard). This report provides details on users’ progress for any lectures or practice activities they might have started or completed across all time or within the time range selected.
Path Insights: this dashboard provides details on users enrolled in a learning path, their progress and if they were assigned or self-enrolled. You can also download this report directly from the dashboard. Learn more about the Path Insights page.
Pro Insights: the Pro insights option allows you to review employee activity within labs and assessments, via the applicable report. Learn more about Pro insights. Note that this is only available to organizations that have Udemy Business Pro for their learners.
Ratings and Reviews Admin Dashboard: Through the ratings and review dashboard you can see how your users have rated courses they’ve taken in Udemy Business, and any reviews they’ve left on the course. Learn more about the Ratings and Reviews Admin dashboard.
Learner Check-in and Learner Feedback: To empower our Udemy Business admins with actionable insights to optimize their learning programs, the Learner Check-in feature helps admin better understand employee sentiment towards Udemy Business. Learn more about the Learner Check-in feature.
Data Export Reports: There are three reports you can download to export data to create your own customized reports inside another tool (the User Activity Report, User Course Activity Report and the User Progress Report). These reports are also available for download from the other dashboards where relevant. Learn more about Data Export Reports.
When is data updated for insights and reporting?
Learn more about when data is updated in your account’s Insights and reporting.