Questions answered by this guide
- How does A11yCheck actually review a course?
- Does A11yCheck read my course file, or the published output?
- What's the difference between how Accessibility, Course Quality, and Course Engagement findings are generated?
- Does A11yCheck use AI, and for what?
- Is a Storyline course reviewed differently than a Rise course?
- What does A11yCheck not check?
- How should I interpret a scan score?
What you'll learn
- A11yCheck opens the real, published course output in a real browser, it doesn't parse the source .story or .rise file
- Accessibility and Course Quality findings are deterministic, checked against a real rule or standard
- Course Engagement findings are advisory, generated with a locally hosted AI model, meant to prompt a look, not a pass/fail
- The same three-pass methodology applies to both Storyline and Rise, only the fix guidance differs per tool
What's inside
Instructional designers, accessibility specialists, and L&D managers deciding how much to trust a scan result
- 01How a scan actually works
- 02Accessibility: what's checked and how
- 03Course Quality: what's checked and how
- 04Course Engagement: what's checked and how
- 05Storyline vs. Rise: what's different
- 06What still needs a human
- 07How to read your results
Why it matters
A tool that won't explain how it reached a conclusion isn't one you can responsibly rely on for a compliance-sensitive review. Knowing which findings are checked against a fixed, citable rule (a WCAG success criterion, an HTTP status code) and which are AI-assisted judgment calls changes how much weight you should put on each one, and that distinction should be stated plainly, not buried in a FAQ.
Example
Reading a mixed-confidence report correctly
✕ Incorrect
A report shows one Accessibility finding (missing alt text) and one Course Engagement finding (a slide flagged as text-dense), and both are treated with the same weight, either both get fixed immediately or both get deferred together.
The alt text finding is deterministic, an image genuinely has no alt attribute, checked against WCAG 1.1.1, not a judgment call. The text-density finding is advisory, an AI-assisted estimate meant to prompt a second look at pacing, not a rule violation.
✓ Correct
The alt text finding is treated as a real defect to fix before publishing. The text-density finding is opened and read with the slide's actual context in mind, timing, audience, purpose, and a real decision is made about whether it needs to change.
This uses each finding type for what it actually is: a deterministic check earns automatic trust, an advisory one earns a second look from a person who knows the course.
Best practices
- Accessibility: runs axe-core (the same open-source engine used across the accessibility industry, 4.x, 30+ WCAG rules) against the real rendered DOM, plus A11yCheck's own checks for things axe-core doesn't cover in an eLearning context specifically, tab order across custom Storyline triggers, hover-only tooltip content, quiz interaction accessibility, and a simulated screen-reader read-through.
- Course Quality: structural checks against the actual rendered course, every link is requested and its real HTTP status is checked, slide and lesson titles are compared for near-duplicates, visual consistency (fonts, button styles) is compared across slides, and known legacy authoring patterns are flagged.
- Course Engagement: word counts and estimated reading time are measured directly from the rendered slide text (deterministic), then a locally hosted AI model (never a third-party API, see the AI note below) adds an advisory read on reading level and whether an interaction looks purposeful or decorative, always labeled as advisory, never scored pass or fail.
- The AI model used for Course Engagement runs locally, not through a third-party API, and only ever augments the Course Engagement pillar. Accessibility and Course Quality findings never depend on it, they run exactly the same with or without it.
Common mistakes
- Assuming every finding in a report carries the same confidence, an Accessibility finding and a Course Engagement finding are generated by fundamentally different processes.
- Assuming a scan reads the .story or .rise source project file, it doesn't, it reviews the actual published output the way a learner encounters it, which is also why a scan can catch things that only exist after publishing.
- Treating a clean scan as proof of full accessibility rather than proof that every automatically-checkable rule passed.
Practical checklist
- Know which pillar a finding belongs to before deciding how urgently to act on it.
- Treat Accessibility and Course Quality findings as close to pass/fail, they're checked against a real rule or a live HTTP response.
- Treat Course Engagement findings as a prompt to look closer, not an automatic defect.
- Still run a manual screen reader read-through before publishing anything compliance-sensitive, a scan is a fast first pass, not a replacement for one.
- Remember a scan reviews the published output, so re-run it after any edit that changes what actually gets published, not just the source file.
How A11yCheck reviews this
This page is the complete methodology; every issue-specific Learning Hub guide links back here for the full picture rather than repeating it.
Frequently asked questions
Does A11yCheck read my Storyline or Rise source file?
No. It opens the real, published course output in a headless browser, the same content a learner would actually see and interact with, and reviews that.
Is a Storyline course reviewed differently than a Rise course?
The same three-pass methodology, Accessibility, Course Quality, Course Engagement, applies to both. What differs is the fix guidance: menu paths, panel names, and known tool-specific gaps (like Rise's drag-and-drop limitations) are genuinely different between the two tools, so the guidance reflects that even though the review process doesn't.
What does A11yCheck not check?
It can't judge whether alt text is accurate to what's actually in an image, whether a learning objective is well written, or whether an interaction is pedagogically sound beyond the purposeful-vs-decorative signal it can estimate. Those need a human. See What Can A11yCheck Review Automatically? for the complete list.
Why does A11yCheck use AI for some findings and not others?
Accessibility and Course Quality findings can be checked against a fixed, objective rule, alt text presence, a WCAG success criterion, a link's real HTTP status. Course Engagement questions, like whether a slide feels dense or an interaction feels purposeful, don't have a single objective rule to check against, so those specific findings are generated with AI assistance and labeled advisory, never presented as a pass or fail the way a deterministic check is.
Related Guides
What Can A11yCheck Review Automatically?→
A11yCheck's automated scan reliably detects structural and technical accessibility issues, like missing alt text, low contrast, and broken keyboard access, but it can't judge instructional quality, learning objectives, or whether an image's description is actually accurate. Here's exactly where that line sits.
See a Sample A11yCheck Scan Report→
This is a walkthrough of an illustrative sample course, not a real customer's course, built to show exactly what a real A11yCheck report contains: how findings are grouped by pillar, what a finding actually looks like, and what the report explicitly does not claim. Run a free scan on your own course to see a real one.
Before You Publish Checklist→
A short checklist to run through before you publish any Storyline or Rise course. It covers accessibility, course quality, and course engagement in one pass.
Course Quality Guide→
What course quality means beyond accessibility: broken links, inconsistent styling, and structural problems that make a course feel unpolished.
Course Engagement Guide→
Why content density and reading level affect whether learners actually absorb your course, and how to review for it.
A11yCheck vs. Built-In and Generic Accessibility Checkers→
Storyline's built-in Accessibility Checker (added May 2025) and generic web tools like WAVE and axe each catch real issues, but neither reviews Rise, neither combines accessibility with course quality and engagement, and neither understands eLearning-specific structure. Here's exactly what each tool does well, and where A11yCheck fits.
WCAG for eLearning: A Practical Introduction→
What WCAG actually is, and what it means for a Storyline or Rise course. No legal jargon.
Accessibility Testing SOP: A Standard Process for eLearning Teams→
A repeatable process, not a one-time checklist, for how a team tests accessibility: who's responsible, what tools and methods to use together, how often to test, and how to document that testing happened.
Course Review Checklist: A Role-Based Process for Reviewing eLearning→
Before You Publish Checklist is one person's fast pass right before hitting publish. This is the process a course goes through to get there: who reviews what, in what order, before anyone signs off. Use this when more than one person touches a course before it launches.
Summary
A11yCheck opens the real, published output of a Storyline or Rise course and runs three passes: deterministic Accessibility checks grounded in WCAG and axe-core, deterministic Course Quality checks against real links and structure, and advisory, AI-assisted Course Engagement findings. Knowing which pillar a finding came from is the key to reading a report correctly.
Review your course before you publish it.
A11yCheck reviews Storyline and Rise courses for:
- Accessibility
- Course Quality
- Course Engagement
www.a11ycheck-dev.com