Workers
The Crowdee crowd panel, worker specializations, and tag-to-pipeline mapping.
Crowdee's crowd panel consists of over one million screened global workers with diverse language skills, domain expertise, and geographic coverage. Workers range from general-purpose content reviewers to certified domain specialists — including forensic analysts, professional fact-checkers, and identity verification experts. Every worker in the panel has passed a qualification process that includes onboarding tasks, accuracy assessments, and policy acknowledgement before they are eligible to receive live tasks.
Worker routing is fully automated. When a pipeline reaches a crowd stage, Crowdee's task distribution system matches the task to eligible workers based on the stage's required tags, the content's language properties, and current worker availability. You do not select, pre-assign, or negotiate with individual workers.
Worker Tags
Worker tags define a worker's area of specialisation. Each crowd stage in a pipeline declares the tags that eligible workers must hold. A worker must have at least one of the required tags to be offered the task.
| Tag | Specialization | Used in |
|---|---|---|
general | General-purpose content review; no specialist knowledge required | verify-image-deep, verify-video-full, verify-social-post, verify-multimedia-cross-modal |
image_forensics | Visual manipulation analysis, pixel-level forensic image review, and artefact interpretation | verify-image-deep |
video_forensics | Deepfake detection, temporal consistency analysis, and video authenticity assessment | verify-video-full |
native_speaker | Language-specific listening and comprehension; workers are matched to the language of the audio content | verify-audio-deepfake |
fact_checker | Journalistic research, source verification, claim assessment, and cross-reference checking | verify-news-article |
identity_verification | Background checks, credential validation, professional record verification, and document authentication | verify-identity-claim |
Tags are assigned by Crowdee during worker qualification. Workers cannot self-assign specialist tags; specialist tags require passing dedicated qualification assessments that are specific to the domain.
Expert Workers
Workers holding the fact_checker or identity_verification tags operate as specialist experts rather than general crowd reviewers. Expert crowd stages use different parameters from standard crowd stages: only two responses are required (minResponses: 2), but both expert workers must agree — the consensus threshold is 100%. If the two experts disagree, the stage verdict is recorded as inconclusive and no further recruitment occurs.
The smaller minimum response count reflects the higher reliability of expert assessments. Expert workers undergo more intensive qualification and are subject to stricter ongoing quality monitoring than general crowd workers.
Expert workers go through additional screening and are held to a higher quality standard. Use expert tags for high-stakes verification where defensibility is essential — for example, identity claims that will inform a legal or compliance decision, or news articles that will be cited in a published report.
Expert stages are slower than general crowd stages because the pool of qualified experts is substantially smaller than the general crowd pool, and expert workers frequently have higher workloads. See Quality Control for a full comparison of expert versus general crowd parameters.
Worker Management
The worker dashboard (Platform → Workers) provides a real-time view of crowd panel statistics relevant to your organisation's tasks, including the number of active workers per tag, current queue depth, average response times, and aggregated IRA scores by tag category.
Worker quality is monitored continuously. Crowdee tracks each worker's historical accuracy against gold questions, their IRA contribution to past crowd stages, and patterns in response quality. Workers who fall below quality thresholds are suspended pending review. Workers who fail repeated quality checks are permanently removed from the panel.
You can view worker statistics and IRA trends from the dashboard, but you cannot view individual worker identities, contact workers directly, or dispute individual worker responses. Quality concerns about a specific run should be raised through the support process at support@crowdee.ai.
No Direct Worker Selection
Task routing is managed entirely by Crowdee's automated matching system. You cannot select specific workers for a task, exclude particular workers, or request a repeat of the same workers who completed a previous stage.
This design ensures impartiality and prevents cherry-picking of favourable reviewers. It also ensures that crowd verdicts are independent — workers assigned to a crowd stage have no prior relationship with the content, the organisation that submitted it, or each other's responses for that task. This independence is a prerequisite for the IRA scoring and consensus mechanisms that make crowd verdicts defensible.
For organisations that require a private or dedicated worker panel — for example, due to confidentiality requirements or the need for workers with specific clearances — Crowdee offers a Managed Workers option as part of the Enterprise plan. Contact sales@crowdee.ai for details.
How is this guide?
Input Data
Input data tables, lists, groups, and variant generation for crowd task customization.
Quality Control
Consensus thresholds, Inter-Rater Agreement scores, and response validation in crowd stages.