RoyaleTrainer: crowdsourcing 180,000+ Image Annotations

The Data Problem

When I started building the Clash Royale AI Vision System, I quickly ran into a fundamental problem: I needed thousands of annotated images to train the object detection and classification models, but annotating that much data myself would take months. Manual data annotation is tedious, time-consuming, and doesn't scale well.

Professional data annotation services exist, but they're expensive and often lack domain expertise. For a game like Clash Royale, annotators need to understand the difference between various units, know what good bounding boxes look like, and be familiar enough with the game to annotate accurately.

The Community Solution

Instead of doing it all myself or hiring annotators, I had a different idea: what if I could harness the Clash Royale community? There are millions of players worldwide who know the game intimately and might be interested in contributing to an AI project.

Thus RoyaleTrainer was born—a web platform where anyone could help annotate Clash Royale screenshots and gameplay frames. The concept was simple: show users an image, let them draw bounding boxes around units and label what they are, and store all annotations in a database for training ML models.

Building the Platform

I built RoyaleTrainer using Next.js for the frontend, which provided excellent performance and a smooth user experience. The annotation interface needed to be intuitive—drawing bounding boxes should feel natural, labels should be easy to select, and the workflow should be fast enough that users could annotate multiple images per minute.

Supabase handled the backend, storing all annotations in PostgreSQL with real-time updates. I implemented features like progress tracking, leaderboards showing top contributors, and annotation validation to ensure quality. Users could see their contribution statistics and how their annotations were being used.

Gamification and Engagement

Getting the first few hundred annotations was relatively easy—early adopters and friends contributed enthusiastically. But scaling to tens of thousands required keeping users engaged over weeks and months.

I added gamification elements: leaderboards for most annotations, achievement badges for milestones, and visual feedback showing how many models were trained using community data. Making contributors feel valued and showing the direct impact of their work was crucial for retention.

The platform also included social features—users could see recent annotations from others, comment on difficult cases, and help establish annotation standards through discussion. This built a sense of community around the project.

Quality Control

With crowdsourced data, quality control is essential. I implemented several mechanisms to ensure annotation quality: consensus voting where multiple users annotate the same image, outlier detection algorithms that flag suspicious annotations, and trust scores for contributors based on agreement with consensus.

I also manually reviewed samples of annotations and provided feedback to contributors. The most dedicated users became unofficial moderators, helping new contributors understand annotation standards and catching errors before they entered the training dataset.

The Results: 82,000+ Annotations

RoyaleTrainer exceeded my expectations. Over 500 community members contributed more than 82,000 annotations over several months. This dataset became the foundation for training the AI Vision System, and the quality was remarkably high—comparable to professionally annotated datasets.

The platform demonstrated that with the right tools and incentives, community crowdsourcing can solve real data problems at scale. Contributors felt invested in the project's success because they were directly involved in building it.

Key Lessons

This project taught me that building tools for others to help you is incredibly powerful. Rather than doing everything myself, creating infrastructure that enables community contribution multiplied my capabilities hundreds of times over.

Transparency and communication were crucial. Regularly sharing updates about how annotations were being used, showing model training progress, and acknowledging top contributors kept people engaged and motivated. People want to feel that their contributions matter.

Finally, gamification works when it's authentic. The leaderboards and badges weren't just arbitrary points—they represented real contributions to a real project. That authenticity is what made them effective.