Embedded Files
Client
Centers for Medicare & Medicaid ServicesDuration
April 2019 - December 2019Role(s)
UX Researcher, Designer, Information ArchitectUX Toolkit
- Quantitative Analysis
- Data Visualization
- Focus Groups
- Personas
- A/B Testing
- Usability Testing
Software Used
- Optimal Workshop (card sorting)
- Google Analytics
- Zoom (virtual focus groups)
Overview
Organizations with complex structures often have equally tangled knowledge management systems (KMS). Without clear information architecture (IA), version control becomes chaotic, orphaned pages clutter the top level page tree, and page maintenance lacks consistency. If you’ve ever bought something at the grocery store only to find it later hiding in the back of the pantry shelf, you know this frustration. But no matter the form, the outcome is the same: wasted time searching for things and maintaining redundancy.
"I don't know where to put the geospatial data. I guess I'll just add it to a new shelf."
In 2019, I joined a 4-person team tasked with implementing operational improvements for a $20 million contract with a branch of the Centers for Medicare & Medicaid Services (CMS) responsible for certifying health plans on the federal health exchanges. Although Confluence was used as a KMS to coordinate annual activities and IT projects, page traffic was low given the organization’s size and the volume of work tracked in Jira.
At the time, UX and human-centered design maturity was low, so I wore many hats—researcher, designer, and information architect. I started by piloting simple navigation updates on select pages to promote open collaboration and shift documentation away from personal drives. My pilot resulted in an 889% increase in page visitors, which led to my pilot growing into a 7-month initiative to adopt Confluence as the primary platform for organization-wide collaboration while improving its navigation and overall usability.
While initial expectations were focused on making pages “look nice,” I used Google Analytics data to make the case for user research and inform the information architecture framework to future-proof the KMS. I led focus groups, defined user personas, and conducted A/B usability tests to ensure the new structure aligned with how different members of the organization actually worked.
The project was a major success, resulting in a significant spike in user engagement within the Confluence space. The new framework also allowed our team to safely expand page creation permissions without compromising design or navigation consistency. Ultimately, the redesigned space became a gold standard for Confluence usage across sister CMS branches.
Google Analytics data showed that even small changes to the navigation flow resulted in significant increases in user engagement. This helped me convey the value of this research to our senior stakeholders.
Because the content in Confluence was highly specific to the organization, I ran a card sort study to uncover how teams mentally organized information in relation to the org structure.
This before-and-after comparison shows how I redesigned the landing page to align with what visitors come to find and what team members need to complete their tasks.
Project Case Study
1. The Project
My clients at the Centers for Medicare & Medicaid Services (CMS) use Confluence as a knowledge management system (KMS) to coordinate annual activities for certifying health plans on the federal exchanges. These activities often require cross-functional collaboration across policy, financial, and technical divisions, each with their own business processes and methods for organizing information. Despite operating on a shared platform, keeping content categorized and easily navigable became an increasing challenge. Users frequently reported difficulty locating documents previously shared via email and often created new top-level pages simply because they didn’t know where to save files—further exacerbating the issue.
2. My Role
Prior to the project, I was an analyst responsible for maintaining Confluence pages that tracked development activities and supported program operations. Most business requirements came through informal stakeholder requests, typically asking for a new page and pointing me to a general location. As requests grew, so did complaints about poor navigation.
I took the initiative to pilot some improvements to our most-visited page, which coordinated IT projects, by adding basic navigation elements to reduce friction. After clients took notice of improved efficiency, this one effort ballooned into a content architecture refresh for the entire space—an initiative I was asked to spearhead given my expertise in human-centered design. My primary role was a UX researcher, but I found myself also acting as an information architect and designer.
These Confluence pages were expected to do a lot for our stakeholders. To avoid the same pitfalls of information disorganization after the refresh, I needed information architecture guidelines to inform how information is presented, organized, and made accessible.
3. The Approach
To improve organization and findability in the organization’s KMS, I implemented an information architecture that catalogues pages and files according to the users’ mental models while designing a navigation flow that aligns with the users’ workflow. It was crucial that this structure was scalable, met the individual needs of each team, and aligned with the way the organization operates.
For the navigation improvements to be effective, the new information architecture and page layouts needed to be consistent across the entire Confluence space. Initially this was a roadblock, because I wasn't authorized to edit all of the pages in the space. To advocate a broader refresh, I made a data-driven case for organization-wide consistency by using Google Analytics to emphasize how consistent navigation and information architecture correlated with higher user traffic and engagement.
Research
Google Analytics
Focus Groups
Card Sort Study
Synthesis
Problem Statement
Personas
Ideate
Brainstorm Sessions
User Flow Diagrams
Information Architecture Diagram
Prototype
Lo-Fidelity Mock-Up
Interactive Prototypes
Test
A/B Tests
Test Case Scenarios
Virtual Usability Tests
4. Research
4.1. Google Analytics
I used Google Analytics to collect user engagement data and build a case for consistent navigation and information architecture across the entire space. By aggregating monthly visitor data at the top-level page hierarchy and then standardizing the values to a normal distribution, I created multiple heat maps to visualize activity patterns relative to average traffic across the space and individual teams. I also overlaid CMS's annual activity calendar to compare page engagement with expected spikes in certification work.
Two key insights emerged from the analysis:
After piloting my navigation improvements in June, traffic to the Project Management Office pages increased by 889%.
Several low-traffic pages showed little to no engagement, even during peak coordination periods.
I coupled these quantitative insights with qualitative data to understand specific dips in page traffic and evaluate the relevance of these lower-performing pages.
The heat map above uses the full range of values, which helps compare activity across the entire space, but it skewed toward pages with deeper, more nested page trees. In these cases, I had to standardize the values across each parent page.
4.2. Focus Groups
Due to time and resource constraints, I conducted focus group sessions rather than individual user interviews. Participants were grouped based on team which helped us map various operational dependencies across the organization. Historical user engagement survey data was used to supplement individual user insights and feedback.
5 Focus Group Sessions
Communications Team
Policy Team
IT Development Team
Operations Team
Compliance Team
Inquiry Topics
The team's roles and responsibilities
How the team uses Confluence to support responsibilities
Other frequently visited pages
How teams interact and collaborate cross-functionally
Common user issues or frustrations
Inactive pages/spaces
4.3. Card Sorting
The client’s organization works with information pertaining to a very specific subject matter. A card sort study allowed me to understand how users categorized their own information. Using the results, I designed a hierarchical taxonomy that aligned with the general user’s mental model.
Using Google Analytics data, I identified 3 to 4 high-traffic pages sampled across each of the top-level parent pages to use in a unmoderated card sort activity.
Dendrograms illustrated relationships between pages and revealed clusters of content that participants naturally grouped together.
5. Personas
I created three, lightweight personas with varying levels of comfort using the system to ground our design discussions. Although each team had a unique use case for the KMS, many of the anecdotes shared during the focus group sessions involved frustrations with navigation and findability.
The New User
Recently onboarded to a team and is new to using ConfluenceGoals
Accomplish tasks with limited knowledge of the scope of work.
Needs
User guides and technical resources
Clear call-to-action text
Pains
Direct links do not help orient the user to where the page is located
Encountering unfamiliar terminology
The Avid User
Uses Confluence daily and collaborates heavily with other teamsGoals
Ensure deliverables are disseminated to the appropriate teams
Needs
A shared space for all teams where folders are organized by annual activities
Pains
Digging through multiple team spaces to find a specific file
Cluttered user interface makes navigation more difficult
The Occasional User
Only uses Confluence to complete seasonal, cross-functional activitiesGoals
Complete tasks without having to review instructions each time they use the system
Finalize deliverables for other teams to use
Needs
A designated space for their team to complete tasks and upload files
Pains
Going to multiple places to upload files
Direct links don't orient users to the page's location on the site
6. Mapping
6.1. Current State User Flows
I mapped the current user flow to identify points of friction. I found that there were too many decision points where the user had to determine whether to look for documents by functional area (team) or by the year. These flow diagrams helped support my case for redesigning the entire KMS’ UI and navigation.
6.2. Future State Architecture Framework
Most users did not visit pages outside of their team’s scope unless they needed something from another team. This site map organizes all content under a general Document Repository (DR) by year but allows teams to locate everything they need within their team-specific pages—called Portals—by using a page tags to pull pertinent files and information into the appropriate space.
By organizing scattered pages under team-specific Portals, we reduced the creation of orphaned pages.
6.3. Aligning User Flows with the Architecture Framework
With the information architecture optimized to provide more flexibility for how files and information are accessed, I then created a user interface that supported this framework. Although files are stored in a separate part of the system, users do not need to navigate outside their team’s pages to find what they need. This provided users the option to locate content from their team's page or by searching in the DR by year. Because the actual content was still saved in the DR, there would be no issues regarding version control.
I color-coded my information architecture diagram to highlight the difference between where pages were stored and where content would dynamically update on user-friendly interfaces.
7. Redesign
7.1. System Limitation
I had to work within Confluence’s limitations since we weren’t authorized to modify the source code. Using only basic page design features and a set of pre-approved plug-ins, I got creative with visual hierarchy principles, color, and navigation elements to improve page readability.
I designed pages that could easily be replicated and repurposed using Confluence plug-ins and templates.
7.2. Minimizing Cognitive Load
The original UI relied heavily on the site map for navigation due to a lack of intuitive navigation elements and a text-heavy layout. Users often struggled to find information saved outside their usual team space. The refreshed homepage UI organizes links by functional area and team, making navigation easier. It also removes rarely used elements (e.g. activity timeline, long list of help links) that cluttered the interface without adding value.
Home Page - March 2019
Home Page - December 2019
8. Evaluation
I ran a moderated remote usability test and an A/B test where I asked participants to complete a list of tasks and compared their success rates using interactive prototypes created directly in Confluence. Test Group B’s results were controlled by only having participants complete tasks in a test environment with the proposed changes.
Test Group A
Complete a list of tasks in the current environment
Complete the same list of tasks in a test environment with the proposed changes
Test Group B
Complete a list of tasks in a test environment with the proposed changes
Research Questions
What do participants from Test Group A think of the proposed changes made in the test environment compared to the current environment?
Are participants from Test Group B able to complete the tasks quicker and with fewer clicks than participants from Test Group A?
9. The Outcome
Thanks to Confluence’s simple page design tools, it only took a month to redesign and migrate content using the new architecture framework. While I initially received a spike in “Where can I find X?” questions, users quickly adapted to the new navigation and user flows. Within three months, requests for custom page creation dropped significantly as users grew confident creating their own pages using the new design and organizational system.
Portals carried cognitive weight, so introducing a new top-level pages became a meaningful business decision rather than a casual action.
Google Sites
Report abuse