How to Conduct a Remote Card Sort Study

Difficulty: Intermediate
Time Required: 2-3 hours (setup) + 1-2 weeks (running the study)

Remote card sorting allows you to collect valuable information about how users organize and understand your content without geographical limitations. By conducting card sorts online, you can reach participants worldwide, collect data automatically, and analyze results more efficiently. This guide walks you through the process of setting up and running an effective remote card sort study.

What You'll Need

• A remote card sorting tool (like Free Card Sort, OptimalSort, or UserZoom)
• A list of content items or concepts to sort (usually 30-60 items)
• Access to your target user group
• Clear research objectives
• A way to compensate participants (optional but recommended)
• 1-2 weeks for data collection

Step 1: Define Your Research Objectives

Begin by clearly articulating what you want to learn from this card sort. Are you trying to create a navigation structure? Understand how users categorize your products? Validate your information architecture?

Write down 2-3 specific questions you want to answer through this research. For example, "How do small business owners group our financial services?" or "What mental models do users have about our content categories?"

Why this matters: Starting with clear objectives helps you design the right study and analyze the results effectively. Without clear goals, you risk collecting data that doesn't address your actual design challenges.

Step 2: Prepare Your Cards

Create a list of 30-60 items that represent the content, features, or concepts you want users to organize. Each card should:

• Contain a single, clear concept
• Use language your users understand (avoid jargon)
• Be at a similar level of specificity/granularity

For example, if organizing a grocery app, cards might include "Dairy Products," "Fresh Vegetables," "Household Cleaners," and "Baking Supplies," rather than mixing specific items like "Milk" with general categories like "Produce."

Why this matters: The quality of your cards directly affects the validity of your results. Too many cards (over 60) can cause participant fatigue, while too few won't provide meaningful insights.

Step 3: Choose and Set Up Your Remote Tool

Select a card sorting tool that meets your needs. Free Card Sort is great for simple studies, while paid tools offer additional features.

• Create an account on your chosen platform
• Set up your study parameters (open vs. closed sort, time limits, etc.)
• Enter your card items
• Configure any additional instructions or questions
• Test the study yourself before launching

Example: In Free Card Sort, you'd click "Create New Study," select "Open Sort" (if you want participants to create their own categories), enter your cards in the card management section, and customize the introduction text to explain your study's purpose.

Why this matters: The right tool makes it easier to collect and analyze data. A poorly configured study can frustrate participants and lead to incomplete or unusable results.

Step 4: Recruit Appropriate Participants

Aim to recruit 15-30 participants who represent your actual users. More participants are needed for remote studies than in-person ones, as some may drop out or provide low-quality data.

• Identify your target user segments
• Create a recruitment screener to ensure participants match your criteria
• Consider using participant recruitment services (like UserInterviews, Respondent, or social media)
• Offer appropriate incentives ($20-50 is typical, depending on study length)
• Send clear invitations with time estimates (usually 15-20 minutes for completion)

Why this matters: The right participants provide relevant insights. Having too few participants (under 15) may not give you statistically significant patterns, while the wrong participants can lead you in misguided directions.

Step 5: Launch and Monitor Your Study

Once everything is set up, it's time to launch your study and keep an eye on progress.

• Send participants unique links to your card sort
• Monitor completion rates daily
• Send reminder emails to those who haven't completed the study
• Check the first few submissions to ensure data is being collected correctly
• Be available to answer questions from participants

Example: "We're seeing that about 60% of invited participants have completed the study after 3 days. We'll send a friendly reminder to the remaining participants, giving them another 4 days to complete it."

Why this matters: Active monitoring allows you to identify and address issues quickly, ensuring you collect enough high-quality data within your timeframe.

Step 6: Analyze Your Results

After data collection, analyze the results to identify patterns and insights.

• Look for common groupings across participants
• Identify items that were consistently grouped together
• Note items that caused confusion (placed in many different categories)
• Compare category names participants created (in open sorts)
• Look for differences between user segments if applicable

Example: "We found that 80% of participants placed 'Monthly Budget Template' and 'Expense Tracker' in the same category, often labeled something like 'Money Management' or 'Financial Planning.'"

Why this matters: Proper analysis transforms raw sorting data into actionable insights that can guide your information architecture decisions.

Step 7: Apply Findings to Your Design

The final step is putting your insights into practice.

• Create or refine your information architecture based on common groupings
• Consider using participants' category labels for navigation items
• Address confusing items by reconsidering their placement or clarifying their purpose
• Document your findings and decisions for stakeholders
• Consider follow-up research to validate your new structure

Why this matters: The ultimate goal of card sorting is to improve your product's usability. This step ensures your research translates into tangible improvements.

Tips and Best Practices

• Pilot test your study with 2-3 people before full launch to catch issues
• Keep card labels concise (5 words or fewer if possible)
• Provide clear instructions at the beginning of the study
• Include a practice example to help participants understand the task
• Set a reasonable timeframe for completion (1-2 weeks typically works well)
• Analyze quantitative and qualitative data together for deeper insights

Common Mistakes to Avoid

• Creating too many cards (over 60), leading to participant fatigue
• Using technical jargon that participants don't understand
• Recruiting too few participants (less than 15) for statistical significance
• Launching without testing the study setup first
• Setting unrealistic deadlines for completion (less than a week)
• Forgetting to follow up with participants who haven't completed the study
• Taking results as absolute truth rather than directional guidance

Next Steps

After completing your remote card sort, consider these follow-up activities:

1. Run a tree test to validate your new information architecture
2. Create a prototype implementing your new structure
3. Conduct usability testing to ensure the new organization works in practice
4. Document findings for future reference and stakeholder communication

Want to get started right away? Create a free account on Free Card Sort and set up your first remote card sorting study in minutes.

Remember, card sorting is just one tool in your UX research toolkit. Combine it with other methods for a comprehensive understanding of your users' mental models and needs.