Bot Detection

Business Objective

The objective of the system is to minimize the negative impact of bot activity on authentic user experience, subject to a strict false-positive guardrail that protects legitimate users from wrongful account restrictions. We're balancing three critical surfaces here: users need protection from spam and malicious interactions, creators need a fair playing field where organic reach isn't diluted by coordinated inauthentic behavior, and the platform needs to maintain trust while operating at scale where even a 0.1% false positive rate translates to hundreds of thousands of real users being incorrectly flagged.

What's particularly important is framing this as impact minimization rather than simple detection. A bot account that sends one spam message before we catch it causes far less harm than one that sends ten thousand friend requests or goes viral with misinformation. This naturally leads us toward a risk-based enforcement strategy where we don't just ban or allow; we have gradations like rate limiting, demotion, and shadow restrictions that let us act quickly even when we're not 95% confident.

ML Objective

At its core, this is a binary classification problem: is this account a bot or legitimate user? But the reality is more nuanced. We need to output calibrated probabilities that map to specific enforcement actions, and we need predictions available at multiple decision points, account registration, during active sessions, and in batch for retroactive sweeps. The model also needs to be robust to adversarial adaptation, which means we can't just optimize for today's bot signatures; we need representations that capture fundamental behavioral patterns that are expensive for adversaries to mimic.